Pete Doe

If you like Steve Martin, me, and/or Main Street Brand Strategy - please check out my newest post, part 2 of my series on what I learned about branding from Steve Martin, then drop us a like, comment or share which would really help us out!

10

Brands are still in the midst of figuring out how to best integrate #AI into their day to day business. In this @Business Reporter article, my colleague @Eda Cetinok dives into Ipsos’ GET.AI framework that helps organizations prepare for AI adoption. Check it out: https://lnkd.in/gdDAVKW5

14

Generative AI can collect responses and analyze them more quickly and efficiently, to glean takeaways quickly and accurately. Move past the “wow” phase and start asking “how can I use this to grow my business?” https://lnkd.in/gnunHz7N Visit our Insights to Activate #GenAI portal to 🎞 Hear how (April 17 webinar) 📚 Read how (Point of View) ❤️ https://lnkd.in/gWhKPjve #insightstoactivate

1

I hear a lot about bots and bad respondents in market research. I'm a big proponent of taking respondent quality seriously because it can impact certain types of studies systematically (e.g., segmentations). We made an automated bad respondent detector that looks for internal consistency in survey responses. The last study we used it on it only detected only 1 completely random respondent (e.g., a bot answering completely randomly). But it still detected 364 bad responses (20% of the data) using a more sophisticated method. We detect based on a combination of low internal consistency in responses and low variation. (All of this is in addition to traditional detection methods like speeding.) For example, someone answering 2s and 3s for everything with no discernable pattern would get caught. This is respondent behaviour that would avoid traditional algorithms for bad data quality. I think either our field partners are catching all the random bots, OR the bots are "smart enough" to answer like lazy humans who use subsets of scales and anchoring. The 20% bad respondents are also likely to be many real humans who aren’t paying much attention. This figure may be conservative, but you have to tweak detectors to be the right balance between false positives (throwing away real respondents) vs. false negatives (not throwing away a bot). But it’s interesting that >99% of what we’re catching is due to behaviour that would avoid less sophisticated methods. Either humans evolved from straightlining to minimal (but random) variation or bots have evolved from fully random variation to minimal (but random) variation. (Our tool uses a bunch of past datasets, both real and synthetically created, extracts features that are markers of internal consistency, and uses a neural network on top of all that to make a final prediction of good/bad respondent. Internal consistency is a relative measure and is derived automatically study to study based on available data. We trained the model to differentiate between high quality real data and randomly created data to match known and expected patterns of bad responses.) Dig Insights, Upsiide, by Dig Insights

49

10 Comments

Many people are talking about how LLMs can't count the number of r's in "strawberry." Most people say it's because LLMs aren't good at math, but that's not the issue. To understand the issue, you need to understand how LLMs work and how the internals interact with the task at hand, which comes down to: 1. Tokenization (how LLMs interpret words) 2. Training data -- Tokenization -- LLMs break words down into tokens. Tokens are small words or parts of larger words, but importantly, they're usually not broken up by letters. Why does this matter? It means they just see a word like strawberry as a single unit as opposed to the letters that make up the word, like we see it. This is efficient for processing (and efficient for training the model). See the screenshot below using the GPT-4 tokenizer. Interestingly, strawberry is a single token when there is a space before it (most of the time) and it is broken into multiple sub-tokens when it either starts a sentence or when a quote comes before it. This tokenization quirk would likely add to its confusion slightly and may be partly why strawberry is picked on. "Blueberry," on the other hand, is broken into "blue" and "berry" tokens in both cases. -- Training data -- It still knows "strawberry" is a compound word made up of the words "straw" and "berry" because that would be in its training data many times (one of those things that just comes up in conversation). But there are too many combinations of words and letters for all combinations to be in the training data enough for LLMs to learn it. Note that LLMs still count the letters in many words accurately; that's because they do have some examples in their training data, and they're pretty good at pattern matching to generalize this to other words.

31

2 Comments

This article has great suggestions on how to incentivize respondents and increase response rate. Virtual Incentives

17

Here we are, another major AI milestone reached: PersonaBot! Imagine being able to load segment profiling data into AI and then have a live conversation with each persona. Welcome to a new revolution in #marketresearch using #AI. This is truly inspiring. Check it out: https://lnkd.in/gGyeaJMW

25

"The most important skill you could build is the ability to ask questions... the smartest person in the room is the one who asks questions, not the one with the answers." - Christopher Frank As a moderator, I love this idea. And as someone who often finds herself thinking "am I the only one who doesn't get this?" I love it even more! .Priscilla McKinney. and Chris make the case that asking questions has always been important, and now it's relevant to AI prompt engineering as well. If you don't know what the question is, how can you get the right answer?? I also love the distinction between summary (organized data) and synthesis (data + judgement). This is a great episode for anyone using data and insights in their role. PSB Insights #littlebirdmarketing #marketresearch

13

1 Comment

Losing survey respondent interest early on can have serious consequences, impacting the quality of responses. One of the worst things a market researcher can do is bore a survey taker, potentially leading to false conclusions. To keep them engaged, consider these tips: - Inform them of the number of questions upfront - Maintain a format that is straightforward but not overly simplistic (avoid true/false & yes/no) - Share the results with them; everyone appreciates seeing their POV! Remember, if you offer an incentive, clearly tell them! Keeping respondents engaged is key to obtaining valuable insights. #MarketResearch #SurveyTips

7

Level up your research game with our ultimate guide, "Everything You Need to Know About Quantitative Research." Uncover the secrets to crafting killer research questions, automating like a pro, and nailing data quality. Get ready to revolutionize your quantitative research - grab your copy now: https://bit.ly/4aPQE1k #MarketResearch #QuantitativeResearch #DataInsights #ResearchTips

7

Level up your research game with our ultimate guide, "Everything You Need to Know About Quantitative Research." Uncover the secrets to crafting killer research questions, automating like a pro, and nailing data quality. Get ready to revolutionize your quantitative research - grab your copy now: https://bit.ly/4cbdX6O #MarketResearch #QuantitativeResearch #DataInsights #ResearchTips

6

The definitions of qual and quant are getting blurred with new AI capabilities as is clear with our new AI moderated video feature in Upsiide, by Dig Insights. But this confusion over definitions of quant and qual isn’t a new phenomenon. Social scientists and statisticians (my undergrad) use the terms quantitative vs. qualitative data to refer to continuous vs. discrete data in a model. For example, if you have a “qualitative” dependent variable, then you’d run a logistic vs. linear regression. In market research (my industry), qual typically means unstructured data (e.g., interviews/text) and quant means structured data, usually survey data. But other data heavy industries sometimes refer to surveys as qual because they're stated opinions and behaviors, not data on what actually happened. To some people, quant means behavioral data (e.g., purchases at store). And in the future, our data capabilities may increase by another step change. We could have real-time data ingestion using computer vision techniques to quantify almost anything, such as how many people are wearing a specific brand of shoes in a specific city. The AI is already capable, but it’s a complex issue in terms of data privacy and security. If that does come to fruition, then maybe the goal posts shift again and current behavioral quant data becomes “qual” and real-time data is the new gold standard “quant”?

15

2 Comments

Johnson's Relative Weights is the most useful statistical tool available for Importance Indexing Brand Equity Measurement, Target Audience Segmentation, Marketing Mix Models, Advertising Campaigns, Content Marketing, Customer Journey-Mapping, Product Line Optimization. https://lnkd.in/e8aYTkyZ .

1

Johnson's Relative Weights is the most useful statistical tool available for Importance Indexing Brand Equity Measurement, Target Audience Segmentation, Marketing Mix Models, Advertising Campaigns, Content Marketing, Customer Journey-Mapping, Product Line Optimization. https://lnkd.in/e8aYTkyZ .

2

Johnson's Relative Weights is the most useful statistical tool available for Importance Indexing Brand Equity Measurement, Target Audience Segmentation, Marketing Mix Models, Advertising Campaigns, Content Marketing, Customer Journey-Mapping, Product Line Optimization. https://lnkd.in/e8aYTkyZ .

1

Johnson's Relative Weights is the most useful statistical tool available for Importance Indexing Brand Equity Measurement, Target Audience Segmentation, Marketing Mix Models, Advertising Campaigns, Content Marketing, Customer Journey-Mapping, Product Line Optimization. https://lnkd.in/e8aYTkyZ .

4

Johnson's Relative Weights is the most useful statistical tool available for Importance Indexing Brand Equity Measurement, Target Audience Segmentation, Marketing Mix Models, Advertising Campaigns, Content Marketing, Customer Journey-Mapping, Product Line Optimization. https://lnkd.in/e8aYTkyZ .

1