Things I learned developing Search Software

Most software starts with someone who is frustrated and is looking for a solution that does not currently exist or if it does, does not quickly solve the problem. The following insights are from 18 years of developing over 100 tools to make it more efficient to complete my projects.

Is there a market for it?

I must say that 90% of the time, there is no commercial market for most tools. During a presentation at SMX Advanced last year, I showed a list of some of the nearly 70 custom tools I have developed to do my job. I use at least a dozen of them on a weekly basis, and of half of these, there is not a commercial equivalent or one that has the specific functions I need. You might ask why, if you need it, the multitudes of SEOs don’t need it. That is always my question: why don’t they need it? It is easy to suggest that we have different approaches. I tend to be data-driven or want to find patterns to speed up the process. The real answer might be below in the “easy button” section.

Here is a prime example: A big challenge for website owners is minimizing the lost traffic from site refreshes and migrations. During these events, sites may lose as much as 80% of their traffic post-launch. With such a negative impact on their business, you would assume that there would be a market for tools that could mitigate this loss.  

We at Back Azimuth were performing many large migrations and acquisition integrations, so we built nearly a dozen tools to facilitate our workflow and demonstrate the potential for loss if quality control was not present.

These applications were not pretty but highly functional. After presenting the methodology at several development and SEO conferences, all the signals showed a significant need for the suite of tools. Who would not want a solution that, if used properly, makes migrations and site refreshes nearly disaster-proof? Given the interest and the size of the business problem, we merged them together, improved the user interface, and launched the Site Migration Suite.

Unfortunately for us, there was/is little to no interest from either the search or the development community. We had a few SEOs who wanted to demo it and, ultimately, about a dozen subscribers who raved about how it saved them from disaster but that it created more work than they cared to do. It was not a lost opportunity since we have made significantly more money fixing migrations at nearly ten times the rate of the tool license.

People want an easy button

I guess it is not unexpected that people want tools to do everyone automatically and present pretty reports. Many don’t want to think or interact with them just want them to do everything and just puke out a successful result.

Of the last 100 emails related to HREFLang Builder, the words that appear most are “automated,” “no effort, and “set it and leave it,” related to the creatures people want to know about the tool.  We get asked almost daily why we have not added AI into the setup to detect the various URL formats a site uses.  We have it, but most users have complex setups requiring human intervention.

People refuse to read user guides

Unless it is painfully easy to set up and use, which is nearly impossible for the first few versions of a tool, you need to assume you will spend a lot of time on setup.

Onboarding has to be painfully simple and as few steps as possible, accounting for any many variances as possible since people will not spend the time to learn how to do it. We are rebuilding the entire setup function of HREFLang Builder since nearly 100% of those who subscribe do not read the read-me-first documents and have problems getting set up.   This requires us to spend a lot of time hand-holding people to get even the simplest of sites set up.

It seems to be such a problem that my friend Dan Weingrod has built a successful consulting practice using Design Sprints to help identify onboarding issues and getting people to set up tools successfully.  

How many customizations do you do?

Nearly every search industry tool developer has needed to build dozens of functions to account for the craziness and dysfunction of what people pass off as websites. For HREFLang Builder, we have over 100 functions integrated to work around issues that violate some of the most fundamental web dev rules. In one regard, this has made it the most powerful tool in the market, but often, we cannot innovate for advanced functions due to dealing with customization to account for a large customer.

We look at customization through benefit tests. If this modification is something that we can use for many customers, we will be a better tool because of it we will spend the time and money to do it. For example, due to the inability of a client to build a complete set of URLs, we needed a method to import URLs from 5 different sources. Initially, we assumed the best source was CMS-generated XML sitemaps but many systems have problems or different methods between CMS. In the end, it worked; it helps clients with complex structures to use the system more easily.

However, if the customization is for a single customer and does not benefit others, then the customer must change their challenge or pay for the customization. We have turned away a few significant prospects who assumed we wanted their business and would do anything to get it. It is often better to say no than take on the risk of developing custom features without a real commitment from the client.

People don’t use the features they request

I assumed this one, but until we added tracking, we could not prove it.   We have done a lot of customization to a few tools based on customer feedback.   We have at least one report in each of the main tools multiple clients have requested that they have never used.  In one case, it was deemed mission-critical to have it, but they never clicked the link to generate it even though they wrote the specification and approved the final live version.   We have been less willing to accommodate feature requests before onboarding a new client. One prospect dangled 200 sites and wanted a custom API connection. We decided to build it so that we could integrate it with other tools. Ultimately, they went with an agency to manually offshore the function.

People don’t want to login

Along with the “easy button,” people don’t want to use tools or log into them.  Several years ago, when I showed him DataPrizm, Conversion Expert Bryan Eisenberg told me to do anything I could to eliminate or reduce the need to log in to the application.  With DataPrizm, using the tool is what it is made for, and just sending reports diminishes the value.  

He was right, especially today. Nearly every user we have now wants us to push reports out to Domo, Google Data Studio, or some internal system. We even had one client pay us to develop an API so they can customize their exports and integrate them into their dashboards and never have to use the tool itself.  

People care too much about pretty over-function. 

I am not talking about something that is complicated to use or lacks basic usability; the feedback is not really what you might expect. Feedback from users, especially on workflow, is critical, but the colors, button placement, and even fonts people want changed are also important. Most of the time, the things they complain about have nothing to do with the actual function of the tool.

We often roll our eyes at people on house hunting shows who walk in and decide not to buy the house due to the color of a room or the carpeting—nothing to do with function, price, etc., but the easiest and least expensive things to change.  

People told us they would not sign up unless we changed the color or removed a logo.   Someone said they had three basic account clients, but before he would sign them up, he required a custom URL and his logo in the tool since he was showing it to clients. Sure, for $75 a month, let me get right on that.  

Who will you upset?

This has been an interesting learning curve with all the tools. In many cases, you would assume people would embrace tools that made their work or lives easier.  It comes down to who and what process you will disrupt.  The more you change a process and highlight inefficiency, the more people will reject the technology.   We have had some negative comments from various consultants and agencies that did not want to use automation, especially for the hreflang functionality they were doing manually.  

One of our biggest detractors for HREFLang Builder was an agency team spending between 20 and 100 hours a month doing them manually for clients.   They did not want to lose those billing hours.  In most cases, the client wanted them to do something more constructive with the time and just license the software. 

Getting Paid 

In the agency example above, you might assume that a tool that costs less than a quarter of the labor hours to do the project would be an easy sell to agencies, but that is rarely the case. From my time at Ogilvy and WPP, getting agencies to pay for tools has always been a problem. They cannot convert billable hours into tool expenses. Most agencies live in a world where they must bill everything to a client, including the cost of or at least a fraction of the tools they use on the client’s behalf. It is nearly impossible to get adopted if it was not budgeted for in the original scope or cannot be directly billed.  

This is often the opposite of how most clients and other rational people think. If you go to get your car serviced by a mechanic, you don’t pay for them to use their tools to fix your car. You expect them to have the right tools and diagnostic equipment to do the job as part of the price you pay for the repair.  This creates a challenge for everyone.

You need flexible payment options either by billing the client or giving the agency access to the solution, which creates a challenge for most subscription and login applications.

So, what to do with your excellent tool idea?

The best approach is to create that MVP and test it to see if people will adopt it. If you don’t get people lining up and raving about it, the market will most likely not adopt it. If it saves you money and gives you a competitive advantage, then build it out for your team to use. If you can crack some of these key challenges when you launch your application, you will be ahead of the curve and have a better chance of success.