I’ve missed more than 9,000 shots in my career. I’ve lost almost 300 games. 26 times I’ve been trusted to take the game winning shot and missed. I’ve failed over and over and over again in my life and that is why I succeed.  – Michael Jordan

Failure is a tricky creature in the business world.  Fail too often and you are likely to be looking for a new job, but the reality is that no one continually finds success without encountering failure.  The problem is that the fear of failure can be so strong that it prevents people from expirimenting, testing, and innovating. 

Companies need to establish an environment where failure resulting from sound business decisions isn’t punished.  Look no further than Google.  Google realizes that not every idea will be a winner but they have created an environment where its employees are encouraged and even expected to constantly innovate – which means trying new things without a paralyzing fear of failure.

What Michael Jordan knows is that if you are too scared to try, you won’t succeed.

As Internet marketers, we love our testing, and one of the greatest benefits of email marketing over direct marketing is the immediacy of testing results.  By ‘pre-testing’, we can use that immediacy to improve the performance of our email campaigns.

A typical A/B test usually involves developing two different versions of an email (e.g. different subject lines, including personalization, etc.), splitting the list of subscribers into two randomly selected groups, and sending a different version to each group.  The test is often run multiple times, results are analyzed, and the information is used to inform future campaigns. 

Most marketers start with what I’ll call ‘macro tests’ which involve larger issues such as testing different layouts, best time of day and day of week to send, etc.  All of these type of macro tests are very important and establish best practices and guidelines for an email program.

However, there are situations in which elements specific to a campaign need to be tested – I’ll refer to those as ‘micro tests’.  For example, maybe the creative director and product manager disagree on which photo should be used in the email as a hero shot or there are questions about the arrangement of words in the subject line (i.e. which are most important to place toward the front).  You could just A/B test the two approaches, sending each version to 1/2 of the list.  However, if one version significantly outperforms the other, then you would have lost opportunity by sending out the worst performing version to 50% of your list. 

Let’s look at the results (similar to one of our client’s recent campaigns) of an email that was A/B tested with 200,000 subscribers and in which version A outperformed version B:

The good news is that we did 20% better than if we would have sent version B to the entire list. However, the bad news is that we performed 20% worse than if we had sent version A to the entire list.  Of course, we didn’t know which would be the best version prior to the send.  Pre-testing allows us to reduce the risk associated with sending a worse-performing email to a large percentage of our list.

A pre-tests involves deploying the initial A/B test to a smaller, but statistically significant percentage of subscribers first and then sending the ‘winning’ version to the remainder of the list.   For example, using the same number of subscribers and response rates in the example above, a pre-test sent to 20% of the list would generate the following results:

In this example, pre-testing improved results by 16% over straight A/B testing.  The greater the performance between the two versions, the more benefit provided (and risk-reduced) by pre-testing.

A few caveats about pre-testing:

• Pre-tests are not suitable for all situations.  For example, there are some tests (like testing a new enewsletter layout) that you are going to want to run multiple times involving as many subscribers in the the sample as possible.  Also, you need to allow at least 24 hours between the pre-test and the send to the reaminder of the list so that you have enough data to reach a conclusion, so if the email is time sensitive, you may not have time for the pre-test.
• Even though you want the pre-test groups to be small, the groups need to be large enough to be statistically significant. (for more on sample sizes and statistical relevance, read Wayde Nelson’s response in a MarketingProf knowledge exchange answer)
• To help validate your approach to pre-testing, run a few tests where you conduct a pre-test with your two versions and then deploy an A/B test to the remaining subscribers.  If you don’t see the same results between your pre-test and full A/B tests, then you need to pre-test with a larger sample size or check to see if something else is impacting results (e.g. day of send).

It’s amazing how often I hear the same misconceptions and misunderstandings from prospects we talk to, clients we work with, or others when discussing Internet marketing.  While businesses have become more savvy with respect to the web, some still have much to learn. 

The following are five of the most frequent Internet marketing myths that I come across:

It’s really about the latest and greatest trend.  While technology continues to amaze, some things don’t change.  Content is still king and in order to engage you have to provide something of value.  The printing press probably wouldn’t have had such an impact initially if Gutenberg had decided to print a book of mutton stew recipes instead of the Bible.  In the end, technology can’t overcome bad design, poor strategy, inferior quality, or terrible customer service. 

I can handle email subscribers in the same was as my direct mail list.  Direct mailers sometimes have a difficult time understanding the repercussions of emailing to subscribers too frequently or to those who didn’t provide clear permission.  Junk mail delivered by the United States Post Office, at worst, ends-up in the recycling bin, while unwanted email triggers spam complaints which in turn tarnishes your email reputation and makes it more difficult to get your communications to those who really want it.  Just because it only costs fractions of a penny to send an email to an email address doesn’t mean that you should.

I’ve implemented XYZ analytics application so I have done all that I need.  Implementing a service like Omniture Site Catalyst or Google Analytics is a great start, but it is just a start.  Companies need to ensure that they have trained staff dedicated to analyzing the information provided by tools like Site Catalyst.  These applications have many wonderful reports but the ‘canned’ reports won’t tell you everything in you need (often they only provide a small portion), and you have to have skilled people focused on analyzing the data in order to gain the insights needed to improve results.

SEO is free – I can just have someone already on staff handle it.  There are companies that handle search engine optimization in-house and are very successful.  But those companies will be the first to acknowledge that it’s not free.  SEO is still such an art and science that you need to dedicate real resources to truly understand how to succeed.  Also, because the rules are continually evolving, a considerable effort needs to be made to stay current and that takes time and commitment.  Companies who think they can buy a book and task someone without search marketing experience to handle SEO in her ‘spare time’ soon discover that approach is a recipe for failure.

I control the conversation.  Companies can foster, facilitate, participate, and even impact the conversation but they can’t control what is being said about their organization.  Customers are having conversations about organizations and their products and services on Facebook, Twitter, You Tube, blogs, Amazon (product reviews), etc.  There is nothing that a company do to stop it and attempts to only backfire.  Organizations must stop worrying about control and learn how to participate.

What Internet marketing myths would you add?

Over the past months, many have proclaimed that the email open rate is obsolete and utterly useless.  The poor open rate has become a persona non grata, and while not as risky as trying to defend the AIG bonus structure, I do think someone needs to stand-up for this email metric.

While I completely agree that clients often focus too much on the open rate, it can be misused, and it isn’t as relevant as it once was because, among other reasons, email clients are more likely today to suppress the image that is used to track an open, I do believe that the open rate still provides value and it is worth following.

Ask any professional football coach what the most important measure of success is and he will tell you that it’s all about wins and losses.  However, he will also acknowledge that when analyzing a team’s performance, you need to go beyond the end result (win/loss) and look at diagnostic metrics like how many yards you gained on offense, how many yards you gave-up on defense, how many fumbles and interceptions you had, how many penalties you incurred, etc.

I believe you take a similar approach with email marketing.  While, if I am an online retailer, I am most concerned about conversion rates and sales, metrics like the open rate provide value.  Obviously, before someone can purchase as a result of your email, they must click on a link, and before they click on a link, they have to open the email.  If you aren’t getting the email to the inbox and the subject line isn’t compelling or engaging, you aren’t going to get a conversion.

I believe that the open rate can still provide insights that will help you improve your email marketing.  If the conversion rate was significantly different between two emails and the open rate for the better converting email was much higher (and assuming the emails were sent within a reasonable time of one another), I might conclude that the subject line, whether it be the way it was written or the offer communicated, was the culprit.  Obviously, I would look at other metrics and analytics but the fact that the open rates were so different would likely impact my conclusions.  Open rates have also been helpful in identifying deliverability issues (when clients didn’t have inbox tracking) and enagement by various segments of a list (e.g. when comparing recent subscribers to ones that have been subscribed for over a year).

So what do you think?  Is it worthwhile to track the open rate or is it ready to go the way of the buggy whip?

We were working with a client to help identify inactive subscribers – those who hadn’t opened or clicked on an email in many months.  The assumption was that these people no longer wanted to receive the email but just hadn’t taken the time to unsubscribe.  We were looking at various changes to the newsletter, and we were interested to see if we could learn anything from this group (e.g. why were they no longer interested in the content and what information might they find more useful) so we decided to send a small segment an online survey as a test.  The first question asked them to rate the value of the newsletter with the expectation that the newsletter would receive low scores.

The survey actually received a good response rate, especially given to whom we sent it, and the respondents gave the newsletter a surprisingly high rating (4 out of 5).  Also, interestingly, a number of people indicated that they hadn’t received an email from the company in long time. 

So, while there were certainly subscribers who were no longer interested in receiving the email newsletter, there were many that still wanted it but a deliverability problem may have been preventing the emails from getting to the inbox.  The overall deliverability to the inbox was very good for this email (as measured by a 3rd party monitoring service), but this information will allow us to work with the email service provider to identify whether there are deliverability issues that our 3rd-party monitoring isn’t discovering.

Therefore, unlike the situations where a friend was listing all of the reasons why a girl didn’t call him back and you had to have that difficult conversation informing him that she just wasn’t that into him, this was a situation in which she was (at least some of them were).

Businesses are always searching for ways to increase revenue, especially in difficult economic times like these.  Often, the suggestion is simiply do more of what is already generating profits.  Email marketing is especially susceptible to this rationale because it is relatively easy and inexpensive to implement compared to other options (e.g. direct mail).  For example, if one email campaign per month generates $50,000 then two campaigns will net $100,000.

However, not only can increasing the frequency of your email campaigns not deliver the expected revenue, but it can result in an increase in unsubsriptions and abuse complaints and a decrease in engagement, especially if you can’t provide relevant content or offers.

In one of his takeways from the 2009 Email Evolution Conference, Chad White recounted the experience of REI who performed a test in which they suppressed emails to non-clickers for 4 weeks, after which they sent them an email promoting REI’s anniversary sale. While the control group was sent several more emails over that 4-week period, the suppressed group (who only received the anniversary sale email) outperformed the control group by 4%.

Does that mean that everyone should reduce the frequency of their email marketing campaigns?  Of course not.  However, it is a reminder that the days of taking a ‘one-size-fits-all’ and one email blast approach to the entire house list should be behind us.  In REI’s case, the company may have identified a group of subscribers who had begun ignoring their emails because they received too many and sending fewer emails (at a lower cost) could achieve as good or better results.

Less can really be more – all it takes is a test (or a few of them) to find out.

When I was in high school, our teacher gave us an assignment to write a speech on a single 3×5 note card.  Even writing real small, it forced us to focus on the most important content and think about each and every word.  A recent study by Epsilon on subject lines reminds us to do the same with email copy.

Epsilon recently released the results of a study on subject line length which supports the generally held notion that shorter subject lines perform better (although the correlation may not be as strong as believed).  Shorter subject lines often do better for technical reasons as many email clients show fewer than 50 characters.  According to Epsilon, 57% of U.S. email recipients see only the first 38 to 47 characters of a subject line when making the decision to open an email.

However, as the Epsilon report notes, the focus on the subject line length itself can camouflage two critical factors for success:  word choice and word placement.  Given the variety of email clients and continued growth of people reading email on mobile devices, the number of characters of a subject line that is visible could be 25 characters or 65.  That is why it is key to place the most important part of your message at the beginning of the subject line and carefully consider each and every word.

Now that doesn’t mean you should never use longer subject lines.  There are many situations in which a longer subject line is warranted and will perform better (the report provides an example).  As such, it is important to continually test elements like placement  (e.g. Ends Today: 10% Off vs. 10% Off Through Friday), word selection (e.g. Ends Today vs. Last Chance), inclusion of your brand, and mentioning specific savings instead of a more general reference (10% off vs. Sale).

At the most recent Email Insider Summit, a panel of moms talked about how likely they are to just delete unwanted emails instead of reporting them as spam. This is relevant to email marketers because we can often look at statistics like unsubscribe rates and conclude that a low unsubscribe percentage indicates that there aren’t any issues with unwanted email.  However, as the panel indicated and others find, unsubscribing may not be the most likely action for someone who doesn’t want to receive emails anymore.  For example, JupiterResearch reports that 26% of consumers unsubscribe using the spam button.

While the more immediate danger is someone reporting your email as spam, which will negatively impact your ability to reach the inbox (many email providers make it the number one factoring in determining whether an email goes to the junk folder), having a subscriber continually deleting your emails isn’t good either.  At the point that someone starts blindly deleting your emails, the subscriber has become detached from your brand and you are not only wasting money sending to those subscribers, but still considering those type of subscribers as part of your core list could skew results and impact your ability to optimize your emails.

As a reminder, here are some of the best practices to avoid subscribers hitting the spam button or just deleting your emails without even reading them:

1. Use and adhere to a good opt-in process.  Sending to subscribers who really don’t want your email is an invitation to be reported as a spammer.
2. Ensure your content is relevant.  For each email sent, you should be able clearly identify what value it provides to the recipient.  Look at alternatives to the mass-blast approach.  Sending the same email to every one of your subscribers makes it more difficult to be relevant.  If you are doing it already, looking at using dynamic content, segmenting, triggered emails, multiple lists, etc.
3. Practice good list hygiene.  Don’t continue to send to subscribers who have disengaged.  If someone hasn’t opened or clicked on one of your emails in a long time (the actual period depends on a number of factors), remove them from your active list and consider some type of re-engagement program to see if you can recover them.
4. Make it easy to unsubscribe.  I know the tendency for some marketers is to make it difficult for someone to remove themselves from a list, but that type of approach will only cause problems as at some point, subscribers will start tagging your email as spam.
5. Don’t look at unsubscribes as the only measure of disengagement.  Realize that a decrease in opens and clicks could be the result of people simply just deleting your emails or marking your emails as spam.  Constantly be assessing whether you are providing something of value that will allow you to continue to keep your audience engaged.

One of my favorite scenes in Woodie Allen’s Annie Hall occurs when Allen’s character, Alvy, is listening to someone in line behind him at the movies expound on Marshall McLuhan’s work.  Alvy disagrees with the man and to settle the point, pulls McLuhan himself from behind a movie cardboard cut-out.  McLuhan informs the man that the man’s interpretation is wrong.  Wouldn’t it be great if we could always go right to the source?  Well, sometimes we forget that we can.

I have been in several meetings with marketers who have been debating the motivations, actions, desires, etc. of their customers, and I am continually suprised in how many situations conclusions are reached and judgements made without any actual data, information, or feedback from the customers themselves.

There isn’t any reason not to be talking to and getting feedback directly from our customers as there are a variety of  approaches that suit all different budget levels.  Online surveys, for example, are an inexpensive and easy way to gather insights that will help better understand the needs and motivations of your customers.  Sometimes, you find that your assumptions were correct and you feel more comfortable acting on them.  However, in other situations, you discover that you were wrong or you learn something completely new. 

You will be surprised at how many of your customers are willing to provide their opinion – all you have to do is ask.

While metrics are an invaluable marketing tool, they also can provide a false sense of comfort.  As an example, your email marketing delivery rate may not be telling you exactly what you need to know.

Email deliverability is obviously critical – if someone doesn’t get your email marketing message, they can’t read it and act upon it.  However, the standard ‘deliverability rate’ provided by many email service providers doesn’t tell you the full story. Often, the number of emails delivered that is reported only reflects those messages not rejected by an ISP.  The deliverability rate may only be reflecting issues such an inbox being full or an email address no longer being valid; it doesn’t tell you how many emails actually make it to your subscribers’ inbox. 

After the ISP (or corporate email server) accepts the email, it may still determine that the message is spam and place it in a junk or spam folder.  In a recent ISP email deliverability study (pdf), Lyris found that Yahoo sends about 26% of messages to the junk folder and Gmail and Hotmail send about 18% each.

You need to make sure you understand whether your email service provider reports on deliverability to the inbox.  If not, you have a couple of options.  One approach is to sign-up for email accounts with major providers (e.g. Hotmail, Yahoo Mail, GMail, etc.) or using existing personal accounts and then send your campaigns to those ‘seed email addresses’ in addition to your regular subscription list and monitor whether the email sent to the seed addresses are blocked, sent to the junk folder or make it to the inbox.  Another approach, which I recommend, is to use a service like DeliveryMonitor.com, Return Path, or Habeas to track deliverability to the inbox.  These services, which vary in cost and functionality, track and report deliverability to the inbox.  DeliveryMonitor, for example, is very inexpensive but still provides results across a large number of ISPs.

If you haven’t checked your inbox deliverability recently, now is the time.