If you’re smart, had an infinite financial cushion and were absolutely mercenary in your choice of market, you could probably choose good markets consistently. [1] Practically, however, most people have to earn money on a day-to-day basis and don't have purely rational interests. How to choose markets in the real world?
Because it takes months of expertise in a market to start generating good ideas, and even longer to turn ideas into opportunities that you can credibly execute, don’t stress about having a perfect product idea initially, and focus instead on choosing a broader market that will be changing by the time you’re an expert in it. Your market choice may not be perfect either, but on average, it’s likely to be ok.
Unlike a specific idea, you should choose a market based on fundamentals, rather than your personal experience. If you choose a market with strong fundamentals, you’ll likely have developed a strong personal connection to it by the time that you’re an expert, since expertise breeds pride. But if you choose a market based on your interests at the moment, there’s no guarantee that it will start changing, which is one of the major drivers of opportunity.
Most people have things they are already good at. Investing, however, requires starting from the basics again, which is humbling. As Ira Glass most famously noted, your taste for good execution will mature before your ability to execute.
Some of the most effective people I know are the first to seek out experts when they encounter a new space. [2] They aggressively look for professors, other entrepreneurs and investors when choosing spaces, similar to Sam, Greg and Elon’s outreach to researchers prior to starting OpenAI. If you’re young, you may not have the career capital to get into meetings with well-known figures [3], but you can typically consume what they write online.
Experts tend to occupy leadership roles and have the time [4], network and experience to make good decisions and provide constraints that can limit your (exponentially large) problem space. Since they are incentivized to provide useful information to the public to grow their careers, their knowledge is also easily accessible - just make sure you stay aware of their biases.
Experts will give you good information but be aware of their misalignment in incentives. The most common biases are portfolio incentive - a leader can make many bets - and proximity bias - people like things that are close to them. Sometimes these biases combine and you get pet projects. Upweight instead what early-career experts are doing personally - their incentives align well with yours.
Starting from experts you can get an idea of what markets might be interesting. Then, it’s time for due diligence - reduce the problem to first principles, run the simulation forwards and question whether the market will indeed be large [5] and changing rapidly [6] at the time that you’ve developed expertise in it.
Don’t be frightened by the anticipated competition - market movements typically leave enough vacuum that startups can fill it if they are nimble at sidestepping strong competitors and relentless in outperforming the weaker ones.
Some important features to consider in a market in 2017:
While startups can be driven by regulatory, cultural or technical change, there’s fewer people who truly understand technical change, and fewer technical leaders who can win the respect of teams that do, so it’s best to bias towards underlying technical change if you have advantages in that space.
Conversely, if you don’t have advantages, keep it simple. Segment and Spotify are not complicated ideas, but they work.
Faster evaluation allows you to change directions more quickly, which is important since your expected value for a market is likely wrong. A new blood test takes a lot longer to understand and try than a new spin on taxicabs, for example.
If it’s hard to captivate your own attention, it will be hard to recruit others or get investment, not matter how lucrative the dollar value of the opportunity is. Focus on markets which have clear, fantastic stories of change.
Contrarian bets on market growth or technical breakthroughs are good for investors (who play iterated games), and bad for individual entrepreneurs (the likely outcome is that you fail and lose career momentum). As an entrepreneur building a career, you need to ship 70% of what you try, so bias towards the past is probably good. If something kind of worked in the past and conditions have changed, it might be worth exploring. However, it's rare that markets appear from nowhere.
Delaying a firm choice delays encountering unknown unknowns, and delays the next (possibly much better) investment. Time is precious, especially in your 20s, so make a firm choice quickly.