Mistakes I Have Made in Product Management

I am a Product guy in the field of AI + NLP or Generative AI. As I go about my day brainstorming, thinking or (sometimes. I am sorry.) pontificating, I often convert some insight about Product Management to a post on my LinkedIn feed. Some suggested that I should compile some of it at a single place. So here is the list, in no particular order. I will keep updating this as we go along.

Last Updated: September 18, 2023

Image by Andre Piacquadio (It’s a model, not me!)

1. The job of a product manager is not to build the greatest products

It’s actually to build great products in the most efficient way.

Not understanding this subtle yet fundamental difference is a rookie mistake. A company is far better off launching good products in the quickest and cheapest way possible, and generating a ton of data on how to improve them, than waiting to launch a perfect product.

(Discussion)

2. Smart (too smart) PMs are best avoided

The Smart-Person-Fallacy astounds me.

I speak with many product managers to hire for my team. They may not be rejected for this reason only, but the number one trait for the candidates who don’t make it is that they make assumptions for their users. “I think the key problem is..”

They make good assumptions, but IMO this is where things start to go wrong in any product. In my experience the greatest products are based on data, research, tests, MVPs, and a lot of hard work rather than sheer genius or a honed gut.

This holds even more true for AI, metaverse and other new technologies, where things are less certain.

To be clear, we need to separate hypotheses, which are data-driven tests that are proven/ disproven based on data, from what I am calling assumptions, which ‘Smart’ PMs just axiomatically take to be true, and that is where they fail.

(Discussion)

3. Good product is not equal to great tech

A good product is NOT a great UX, deeply researched features and an amazingly built tech solution. This is almost a truism in the product world, yet so many people fail to grasp it.

A good product is a mind-blowing experience for the end user that solves some real emotional need.

Sure, that experience may require, for its execution, a great UX, deeply researched features and an amazingly built tech solution.

It actually may not, or can be a great product even if some of these things are not there.

(Discussion)

4. Don’t be fooled by Google Analytics when choosing your product’s form factor

If 70% of your traffic is from desktops should you design your product desktop-first or mobile-first?

This is up for some debate in the world of product management but I am definitely in the school of mobile first designs. Here’s why (the most important reason the last):

• Transferring mobile designs to desktop is far easier than vice versa. Worst case, just add a nice graphic for wider screens.
• Data often suggests that users trying to access a desktop-first product via mobile devices contribute disproportionately to user churn.
• This becomes a self-fulfilling prophecy — you design for desktops, you get only desktop users, and lose out on a huge customer persona as well as many many smaller jobs that the product could do.
• Irrespective of anything else, this 30% is only going to grow. There is no debate anymore that more and more people are moving to mobile devices.
• When you design for mobile you are forced to simplify the UX for a limited real estate, and that always is a better idea, no matter what your product is.

Of course, there are many exceptions to this rule, but I always recommend thinking mobile first.

(Discussion)

5. Defaults are ever-so-powerful

Have you ever given in to the power of default settings?

In product management I have seen too many people make the mistake of assuming that because something is configurable, users will configure it. “We are giving the user the control.” “We are making this personalized.” “We are listening to what the user wants” and so forth.

The truth is that an average user is too distracted, too uninterested in your product, and too confused with it to actually care. They will always start with the defaults. And that’s where the power of defaults comes in. Bad default settings automatically mean bad user experience for an overwhelming majority of your users.

Year or two ago I would have said, please preset the defaults in a way that makes your product the most powerful, and help your hapless user onboard first, before they can eventually become an advanced user and actually leverage these settings.

It’s good advice even today. With AI, though, the bar is higher. Now I would ask, can you auto-configure for each user without them lifting a finger, literally, and without you breaking the rule of sign up to first value within three clicks?

(Discussion)

6. High cost does not always mean unviable products

Pricing a product has nothing to do with its cost. Viability is such a misunderstood concept in product management.

Every PM thinks about Desirability, Feasibility and Viability of a product. The first two are about whether we can build something that our customers will like. Viability is about whether we can build a profitable business out of it. For this, unseasoned product managers start thinking cost plus — if this costs us $80, let’s price it at $100. And that is the root cause of the misunderstanding.

Few things to keep in mind:

1. Pricing has nothing to do with cost. It has to do with the value to the customer. Sound obvious, but this is the most misunderstood idea here.
2. Pricing is also a lever to define the customer persona. E.g. Google targets everybody by making its search free, or many premium brands deliberately price higher and then discount instead of pricing lower to begin with.
3. Viability is not about whether price is more than the cost today. Viability is to ask how can the cost be taken to a point below the price. It could mean developing better monetization strategies without increasing the price, scaling up, rearchitecting things, etc. A business is not viable only if there is no such path (which is rare in SaaS, tbh).
4. Even for a viable product with stable revenue streams, a PM must ask for more: Are there adjacent space or sub-groups within the target persona which could fit the product? How does one price the product to target them? Are there versions that the PM must create? Etc. Just because something is profitable does not mean that it cannot be more profitable.

The question of viability is becoming even more important in the era of LLMs. They are expensive, but that does not mean that their application is unviable.

(Discussion)

7. Everything save real customers is useless when building credibility

In the world of enterprise products there are as many snake oil sales people as people with genuine solutions that help clients. Amid this chaos, how does one differentiate, esp. in the beginning, without a brand?

Of course, it’s not easy, but as a product manager I have found the following to work, in order of effectiveness:

1. Client Testimonial on Video — A client, ideally a CXO or XVP, speaking about your product for 60 secs, esp. talking about how it helped them, is probably the most effective. A client doing the same in a conference or another thought leadership event is better, even if not scalable. Make sure to get this on video.
2. Live Product — A version of your product that a prospect can actually play with, perhaps for their own data/ situation also builds confidence like few others. It’s super important that the process from sign up to value is self-serve and involves as few clicks as possible (max three?). Otherwise this starts to lose credibility.
3. Video of Your Product — A video of your product being used by your marquee client can be helpful, too. Note this is different from an explainer video. Here I mean a video of your product in action for some other customer, ideally multiple customers esp. useful when option 2 is not really available.
4. Case Study with Client Logo — A case study, ideally accompanied with joint PR, where the client talks about how great your product has been, is a much easier get than option 1. Always ask for some press release, or at least a blog or announcement on the client website that you can link to.
5. Client Logo/ Testimonial/ Case Study — Client logos always help. If they don’t allow you to publicly use it, then at least negotiate the use in private settings. In my experience, anonymous testimonials or case studies are not that credible, unless you are someone with established credibility already.

While everything helps, collateral beyond this does not add that much value.

(Discussion)

8. The 2x2 prioritization is all wrong

Have product managers been using their 2 x 2 prioritization framework wrong all this while?

We all know the Effort vs. Impact plot. I meet many PMs who use this framework as follows:

1. Start with low-effort-high-impact things. Of course!
2. Triage between high-effort-high-impact and low-effort-low-impact things. This is where they claim a good judgment comes in handy.
3. Touch the high-effort-low-impact things only in the end.

This seems to make sense, but I have learnt over time that we should not be doing high-effort things, period. Rather we should break them down into various low-effort things with varying degrees of impact.

Then you are back to sequencing between low-effort-high-impact tasks and low-effort-low-impact tasks. I have found that rarely any project or initiative is so atomic that it cannot be broken down. The biggest advantage of this approach is the quicker feedback cycle.

Perhaps this is where true product skills shine.

(Discussion)

9. Prioritization in the face of deadlines can be a whole different game

One thing no course on product management teaches us is prioritization in face of deadlines.

It’s all well and good to talk about the Effort-Impact 2x2 in an ideal, academic world, but when the rubber hits the road, there always are deadlines and other extenuating pressures that make this classic framework less useful.

So how do we prioritize with such pressures? Here is how I encourage everyone, not just product managers, to think about it:

1. Prioritize without the deadline. As a starting point, acknowledging that this may not be the right answer, prioritize the task as if the deadline did not exist. How would you have approached this? When? Most importantly, will you even pick this up within the next 3–6 months if there was no deadline? If the answer is no, then don’t pick it up now. Set the right expectations, manage proactively, align interests — this may come as a surprise, but most of your internal and external counterparts are actually pretty reasonable people!
2. Assess the cost of the exception. The most important question while prioritizing that many people don’t ask is: What will get deprioritized? We can focus on only a few things at a time, so if we are taking something up, what are we not doing, or pushing out? If that thing is more important, then you have a choice to make. Are we ready to pay the opportunity cost in terms of lost sales/ upsells, commoditization and (lost forever) first mover advantage for the initiatives you will push out to accommodate a near-term task?
3. Plan for more chaos. Let’s say stars still align, and you want to take this task up as an exception. Is the cost of this small near term victory a lot of challenges in the longer term? In my experience this kind of ‘exception-handling’ pushes multiple other things closer to their deadlines in a vicious cycle. If that’s true for this task, what’s your strategy to handle the chaos that is likely to follow? Hoping that it will all work out can be good sometimes, but not sure how effective that is. This planning will highlight the true choices you are making.
4. Plan for less chaos. The real question in all this, another one that is very frequently missed, is: How did we end up in this situation? Did we not know the deadline already e.g. did we not see this customer or that regulator was about to put their foot down? What can we do differently, esp. if there is a pattern around it? Are the so-called ‘important’ things that we are focusing on day-to-day not effectively addressing and pre-empting these ‘urgent’ things? They should, and you must pick those ‘important’ things accordingly.

I have often gotten the feedback that this is a bit theoretical — who can think and plan through all this in a high decibel call — but I beg to differ.

(Discussion)

10. Agile Sprints are not two week long

Should a sprint be two weeks? A big resounding NO!

So many product and engineering managers seem to get this wrong. They think a sprint is just a unit of time — two weeks. The argument is that a two week cadence seems right for all the scrum ceremonies e.g. planning meetings and retros. I have often seen people talk about projects as being two-sprint or three-sprint — with the first deliverable being six weeks out — because that’s how long product/ engineering teams think it will take to deliver on it.

This is the wrong thinking.

One of the core tenets of the agile methodology is about responding to change rather than being sticklers to a plan. The idea is that at the start of the project there are many unknowns. If there aren’t then we are starting the project too late. This is truer for AI related projects. Chief among these unknowns is customer value, but they could also be around tech feasibility, partnerships, costs, etc. In short no plan is going to guarantee success unless we set interim deliverables, test them and keep changing the plan as we keep learning more.

The right way to think about sprints is the amount of time it takes to reach an interim milestone that can be tested. This milestone either validates the current plan or changes it. If drafted well, the plan will hit the most ambitious assumptions first and more certain things later.

It just so happens that most organizations think that a two week period is right to deliver something testable. In many cases, organizations acknowledge that the complexity they are dealing with is different and choose to adopt a three or four-week sprint. In others, like fast growing, early-stage B2C startups, things are moving so fast that they work on a weekly sprint.

While two weeks is a good assumption, my suggestion would be look at the complexity of your stack, your business needs and the story point velocity of the engineering teams to decide for yourself. I have also seen situations where different teams adopt different sprint cadence.

TLDR — A sprint is not a unit of time; it’s a cadence at which a product/ engineering team can deliver on well-defined milestones.

(Discussion)

11. Switch off the vitamins, unless there is a penicillin at the end of the road

Should you shut your company down if your product is a vitamin and not a painkiller?

Look — the answer based on conventional wisdom is yes. And I agree with the conventional wisdom on this one. However, if you are the stubborn type, there may be ways to win this game.

One of them is to make the vitamin free for your users and build a sizable user base that loves you. Then multiple things can happen:

• Google model — you can find other ways to monetize the user base.
• Slack model — you can build freemium features that are pain killers
• Facebook model — you can get something else (data) from your users other than money.
• GitHub model — you can train massive model without actually taking anything from users.

The underlying factor is this — if you build things that users love, there is likely to be a way to build a business on the back of it.

(Discussion)

12. Be wiser about competitors’ PMF than customers or the market

Source: Praful Krishna. Feel free to use as long as you link this article.

How do you know whether a competitor has achieved the ever-so-elusive Product Market Fit or not?

It’s actually simpler than you think. Over time, a product without PMF just keeps adding more and more features. It starts to look more and more like a platform or (as the joke goes) a Swiss Army knife. More and more the backers of the product will start touting it’s one-stop-shop abilities. Don’t get me wrong — there are many great products that are one-stop-shops by design; this is different.

If it gets PMF, otoh, the feature set starts to deepen towards a small set of use cases. As the legendary Clay Christensen taught me — every product has a job to be done. A product with PMF goes deeper towards that particular job, before it adds features for adjacent use cases. It’s simply gravity!

This kind of research can really help a product manager design their own products better for competitive differentiation. The best part? This intel would typically come far earlier than announcements of raises or other usual indicators that a company has found a market fit for its products.

(Discussion)

13. Prioritization for AI related projects is far more complex

Source: Praful Krishna. Feel free to use as long as you link this article.

AI is challenging some of the fundamentals of product management.

Remember the good, old 2 x 2 prioritization framework? With AI there are new dimensions that must be added to it. Impact and Cost are what they always were; and even the way to measure them does not have to change. However, in the age of AI, product managers need to think about three more things:

1. Time to Launch: Time has always been super important in the world of product. The sooner you start getting feedback, the better off you would be in your attempt to find the product market fit. With AI, however, time takes a whole different importance. First, the sooner you start generating real world data, the stronger your models will become. Second, and perhaps more importantly, the sooner you can confirm the fundamental assumptions about your modeling, the less likely would you be to go in the wrong direction. For traditional software, time was more or less directly proportional with cost, but not so with AI, e.g., compare GPU-intensive but well understood use case like image classification with, say, a bespoke system to tag leads.
2. Solution Inaccuracy: Unlike deterministic programing, AI comes with the fact that it may not always work. So would you prioritize a quick solution that is right half the time, or a complex solution that is right 90% of the time? The answer is not trivial, ever, but this dimension now is a strong consideration in terms of picking and choosing projects. The kicker? For AI systems you may not even know its accuracy in advance. You can guess, at best. This is why many software teams still prefer predictable lower impact legacy solutions over AI. This is also why quick, agile implementations with lower time to launch become instrumental.
3. Cost of Failure: The final dimension is to think about what happens if the AI gets it wrong, even for highly accurate systems. In a B2C app with a quick feedback loop it doesn’t really matter, because the user will immediately provide a corrective signal and things will be fine. For example — the text auto complete feature in Gmail. Otoh, this can get very tricky for enterprise applications in regulated contexts. A wrong decision can expose the company to law suits or significant financial losses. For example — healthcare, banking and/or trading systems.

So, instead of the 2x2 it’s time to consider the Spider Chart while prioritizing. Plot everything and work from the center outwards. It will be very interesting to see how PMs navigate the inherent trade-offs.

(Discussion)

If you want you can keep following this article or my LinkedIn feed for more sadistic pleasure from my continued mistakes!