by Michael Blanding
Before launching new products, entrepreneurs are often filled with doubt: Will their ideas successfully take off in the marketplace—or will they fall flat? To cut down on uncertainty, creators can post their inventions on platforms such as Product Hunt, where early adopters examine and beta-test new apps and other products, offering feedback to help entrepreneurs refine their ideas.
However, there’s a caveat to this feedback: 90 percent of users on Product Hunt are men, according to a recent working paper by a trio of Harvard Business School researchers called Biased Sampling of Early Users and the Direction of Startup Innovation.
“You’re missing out on information from a huge swath of the population,” says Rembrand Koning, an assistant professor in the Strategy Unit, who wrote the paper along with Sarofim-Rock Professor of Business Administration Ramana Nanda and postdoctoral fellow Ruiqing “Sam” Cao.
That reality is particularly problematic for female entrepreneurs, who are more likely to invent products that address the needs of female consumers. “You could imagine those products being discounted because the men on this platform often lack the lived experience to value the potential appeal of the product,’” Koning says.
This gender gap is hardly limited to Product Hunt. Men also comprise 75 percent of visitors to Kickstarter, 67 percent to Indiegogo, and 79 percent to Y Combinator’s Hacker News. And of course, they dominate the conference rooms of venture capitalists and tech companies that are making decisions on investing in new products. “When you are trying to grow a startup, the people you are getting advice and feedback from are overwhelmingly men,” Koning says.
That phenomenon, known as “sampling bias,” is likely less a result of active discrimination against women, Koning says, than it is a lack of representation in the sample of people giving feedback. Nevertheless, it could have potentially serious consequences for a product’s success.
Female-focused products experience less growth
To test that effect, the researchers examined some 6,000 products released on Product Hunt in a two-year period between 2016 and 2018. Using machine learning, they analyzed product descriptions to classify products according to how much they appealed to female consumers.
An app that allows pregnant women to request a seat on public transport, for example, was rated more than 99 percent female-focused, while an app using artificial intelligence to help managers with diversity was rated at 75 percent. “You can imagine that given the barriers women face in labor markets, they would want managers to use it more,” Koning says.
When they examined product performance over time, they found that on a platform where nine-in-10 users are men, typical female-focused products showed 40 percent less growth than male-focused and gender-neutral ventures a year after launch. In addition, the female products had fewer users and lower rates of venture funding.
Of course, one explanation for the discrepancy could be that female-focused products just sell worse overall. Koning and his colleagues, however, used a clever technique to test that assumption. Product Hunt sends out a daily newsletter, in which it features previously launched products. The researchers figured that on days in which the newsletter featured women-centered products, more women would be drawn to click on the link to bring them to the site. “They might tell their friends about it, and share it on Twitter or Instagram, so on those days you are getting a lot of female engagement,” Koning says.
Sure enough, when they examined the success rates of products on those days, they found growth and venture funding for female-focused startups were just as strong as their male-centered competition. Koning surmises that on those days, founders received positive feedback that not only helped them refine their product, but also made them made them more apt to overcome discouragement and stick it out in the marketplace. “They are likely to keep putting effort into it,” says Koning. “In essence, they are getting a better sample to estimate their product’s potential.”
Entrepreneurs need feedback from their target audience
The study, says Koning, demonstrates how important it is for entrepreneurs to seek out feedback from early adopters who are demographically similar to their target users in order to accurately gauge the market. “If Product Hunt doesn’t go well, then maybe find other channels, build out focus groups, explicitly ensure that you are engaging with those people,” Koning suggests.
While the study only looked at women, the findings could apply to any group that is underrepresented in test audiences for tech, he adds, including people of color and rural consumers. Some venture firms have taken the initiative to specifically seek out products appealing to those underrepresented groups, such as X Factor Ventures, which focuses on women, and Harlem Capital Partners, which focuses on people of color, looking for an edge with products that other players may miss.
Ultimately, however, the product testing world is a reflection of the wider world of tech, which is dominated by white, urban, professional men. To truly recognize the potential of products appealing to women and other groups, Koning says, companies must continue the hard work of diversifying their personnel.
“At the end of the day, the reason most of these platforms are dominated by men is because all the venture capitalists and engineers and other founders are mostly men,” Koning says. “If you want a platform that is more diverse, then you need the larger ecosystem to be more diverse.”