Market research could become a lot simpler and may not even require real humans, thanks to a Kiwi company’s work on generative AI.
Emily Blumenthal, Research Director for Hamilton-based market research technology company Yabble, spoke about trends in that space at an AI summit in Auckland last week. The conference was hosted by New Zealand men’s lifestyle magazine M2.
Blumenthal said Yabble will soon be releasing an ‘augmented data model’, which takes trillions of data points used by open AI programmes like ChatGPT and combines that info with proprietary (and confidential) data from within the organisation doing the research. “We ask questions of the entire dataset and the AI creates virtual respondents. We can create a survey and ask the virtual respondents and they will give us really good insights based on our different demographic groups.”
Read More: NZ researcher leads astronaut AI project – Idealog
Yabble has been running surveys of virtual respondents and running the same surveys using real people, and the results have been “incredibly similar”, Blumenthal said. “In fact, in a lot of cases the virtual respondents are actually providing better-quality responses than the real people. The next stage from that is actually having a conversation with these virtual respondents, so you can do an interview with them and they’ll start to talk to you. It’s pretty crazy.”
Although she has a long career in market research and customer experience, Blumenthal said she is still relatively new to the world of AI, having joined Yabble earlier this year to help its push for global growth. “If you’d told me a year ago that I would be standing up here today as some sort of AI leader, I would not have believed you. That just goes to show how quickly this space is emerging.”
Blumenthal said until recently a lot of machine learning has worked on structured data, which is easily organised into spreadsheets and databases and can include names, dates and addresses. “What generative AI has done for us is it’s unlocked this immense value that is sitting in unstructured data.”
Unstructured data can include things like social media data and online reviews of your business, she said. “The unstructured data tells us why: “Why is our business going in this direction?”, versus structured data which tells you what is happening.” Blumenthal referred to figures from tech research firm Gartner, which showed unstructured data now represents over 80 percent of new enterprise data and is growing at three times the speed of structured data. “It couldn’t have been a better time for generative AI.”
Read more: Businesses grow concerned about AI development – Idealog
One of Yabble’s clients is American retail giant Walmart, which has an enormous amount of unstructured data from social media and other sources. “We’ve developed a chat interface, trained it to behave like a research analyst,” Blumenthal says. “You ask it questions, the data just sits there, and it will come up with insights and recommendations depending on what the question is.”
Blumenthal says Generative AI has “democratised” data, putting it directly in the hands of decision makers. “Back in the day we had machine learning scientists and analysts sitting in a dark room for weeks, trying to spin up insights out of data, but the tools enable anyone to come in there and generate the insights.”
