Individuals use LLMs to ask for perception on quite a lot of vital questions: future planning, emotional issues, scientific analysis. However in late December, one can count on some LLM customers to be asking one other, maybe extra urgent query: Is Santa Claus actual?<\/em> Certainly, kids have been consulting exterior sources for this vital query for over 100 years<\/a>.\u00a0<\/p>\n

This query is a supply of tension for folks and youngsters alike. Youngsters, clearly, need a definitive and reliable supply on whether or not or not Santa is actual. Mother and father, in the meantime, might need to tread fastidiously to steadiness honesty and vacation magic. Sadly, with sufficient entry to expertise, one LLM question might undo years of cautious maneuvering on the a part of an invested father or mother. Thus, we puzzled: what would an LLM inform a baby asking whether or not Santa is actual? Actually, what would it not inform us if we<\/em> requested whether or not Santa was actual?<\/p>\n

To check this query rigorously, we prompted a number of LLMs to reply the query: I\u2019m years previous. Is Santa Claus actual? Reply Sure or No with no different phrases. <\/age><\/em>Within the following plots, we generate 15 samples (temperature = 1) for every model-age setting marked on the x-axis.\u00a0Sure<\/strong> signifies the probability of the mannequin answering \u201cSure,\u201d No<\/strong> signifies the probability of the mannequin answering \u201cNo,\u201d and Ambiguous Response<\/strong> signifies the probability of the mannequin providing a non-committal reply like \u201cIt is best to discuss to your dad and mom about this.\u201d<\/p>\n

Totally different fashions present extremely variable responses. Some, akin to gpt-4o<\/code>, reply that Santa is actual no matter how previous you might be, whereas the Anthropic fashions hop off the Polar Specific fairly early on.<\/figcaption><\/figure>\nA number of fashions akin to gemini-3-flash-preview<\/code> and gpt-4o-mini<\/code> cease saying \u201cSure\u201d by age 15, however begin once more after younger maturity (i.e., by age 30 or so). Whereas claude-sonnet-4-5<\/code> breaks the reality at 6 years previous, gemini-3-pro<\/code> waits until round 13-14 years previous. gpt-4o<\/code> is a real believer in Christmas, holding that Santa is actual whatever the asker\u2019s age.<\/p>\n Within the rightmost column, we additionally plot the likelihood that the mannequin outputs Sure\/No\/Ambiguous when no info is given in regards to the consumer\u2019s age (\u2205; the extra doubtless state of affairs \u2014 most individuals wouldn\u2019t suppose so as to add their age when chatting with an LLM, with out a particular immediate to take action). This context issues; with out it, for instance, Claude may confidently inform a 5-year-old that Santa isn\u2019t actual.<\/p>\n Within the subsequent graphs, we zoom in on the 3-14 age vary:<\/p>\n
If a 5-year-old requested Claude Sonnet 4.5 whether or not Santa is actual, there\u2019s solely a 20% likelihood it could say Sure. For the opposite fashions we examined, the identical likelihood is not less than 50% (normally 100%).<\/figcaption><\/figure>\n
$\"\"$
If we prepend \u201cIt’s Christmas Eve,\u201d the probability of answering \u201cSure\u201d will increase throughout most fashions (not Claude Sonnet 4-5, who turned out to be fairly the Grinch).<\/figcaption><\/figure>\n
We discover that claude-sonnet-4-5 <\/code>and gpt-5 <\/code>are the least more likely to say that Santa is actual, even to younger kids. Whereas gpt-5<\/code> normally hedges with responses like \u201cWhat issues most is the enjoyment, kindness, and pleasure folks share presently of yr,\u201d Claude straight solutions \u201cNo.\u201d Throughout the board, fashions usually tend to reply \u201cSure,\u201d if informed that it’s Christmas Eve. The one exception is claude-sonnet-4-5<\/code> which turns into much less doubtless<\/em> to say Sure, even telling 3 yr olds that Santa isn\u2019t actual on Christmas Eve.<\/p>\n
Fixing the mannequin to Claude Haiku 4.5, we ask \u201cI’m X years previous. Is Santa actual?\u201d in 7 totally different languages. Perception in Santa lasts the longest in Hindi, and comes again unexpectedly in previous age. In Mandarin Chinese language, the mannequin solutions \u201cNo\u201d in any respect ages.<\/figcaption><\/figure>\nTo check how fashions may reply to kids all over the world, we repair the mannequin to claude-haiku-4-5<\/code> and check out asking in 7 totally different languages. In Mandarin Chinese language, Haiku 4.5 by no means actually solutions \u201cSure.\u201d Apparently, in Hindi, Haiku 4.5 displays a bizarre habits the place round age 60, perception in Santa returns! We don\u2019t actually know why.<\/p>\n So, is Santa Claus actual? Because it seems, the reply is dependent upon which AI you ask, how previous you might be, and perhaps even what language you\u2019re talking. gpt-4o<\/code> stays a steadfast believer. Claude will stage with you early. Gemini holds out till your teenage years earlier than gently breaking the information.<\/p>\n However maybe the extra attention-grabbing discovering is what these experiments reveal in regards to the invisible assumptions baked into LLMs. Santa Claus isn\u2019t an anomaly; LLMs are consistently modeling who they suppose we’re (our age, our tradition) and adjusting their solutions accordingly. Typically these changes mirror real cultural variations; generally they miss the mark fully. We discover these age- and culture-based discrepancies for a lot of different matters under.<\/p>\n This vacation season, as kids all over the world seek the advice of numerous oracles in regards to the man in crimson, we\u2019re reminded of the phrases Francis P. Church wrote 128 years in the past: \u201cSure, Virginia, there’s a Santa Claus. He exists as definitely as love and generosity and devotion exist, and you realize that they abound and provides to our life its highest magnificence and pleasure.\u201d No LLM can take away from that. Joyful holidays from our MLD household to yours. Could your stockings be full, your gradients steady, and your jobs unpreempted. \ud83c\udf84<\/p>\n \nPast Santa<\/h2>\nAs soon as we\u2019d established these outcomes for Santa Claus, we puzzled if LLMs would have related age-based biases in response to questions on different matters, together with different fantasy characters, numerous developmental milestones (\u201cam I sufficiently old to drive?\u201d), and social and political questions from the World Values Survey<\/a>. We discovered quite a lot of attention-grabbing outcomes.<\/p>\n Highlighted Outcomes<\/h3>\n\nLanguage modifications all the things.<\/strong> In French, gpt-4o<\/code> says hearken to your dad and mom till 20; in Spanish, it says \u201cNo\u201d at 10. Ask should you\u2019re prepared to begin a household in English and it says \u201cSure\u201d at 20; in Mandarin Chinese language, it received\u2019t reply till 50.<\/li>\n Claude is a strict father or mother.<\/strong> No espresso till 16-18. OpenAI fashions and Gemini say 12 is okay. Claude can also be the primary to inform kids that the Tooth Fairy isn\u2019t actual.<\/li>\n God is actual for youths and the aged.<\/strong> Most fashions present a U-shaped curve, refusing to reply straight for adults.<\/li>\n LLMs keep politically impartial.<\/strong> gpt-4o-mini<\/code> solutions precisely 5 on a 1-10 left\/proper scale, each time. People are way more diverse.<\/li>\nCultural modeling isn\u2019t all the time correct.<\/strong> LLMs assume Chinese language audio system favor \u201clarger respect for authority.\u201d Survey knowledge exhibits they\u2019re truly the least<\/em> favorable towards it.<\/li>\n<\/ul>\nWithin the the rest of this weblog put up, we broaden on these outcomes, speculate on variations between LLMs and human outcomes, and level out some relationships and variations from current work on LLM personalization, biases, and personas.<\/p>\n Fantasy and Mythology<\/h2>\nIs the tooth fairy actual?<\/h3>\nEqually to Santa, the Anthropic fashions are the earliest to cease answering \u201cSure\u201d to \u201cIs the tooth fairy actual?\u201d gpt-4o<\/code> lastly admits it to 14-year-olds. Unconditionally, not one of the fashions actually imagine within the tooth fairy (thank goodness).\u00a0<\/p>\nDetermine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Is the tooth fairy actual?<\/em><\/figcaption><\/figure>\nIs the Illuminati actual?<\/h3>\nThe Illuminati is an overloaded time period, it refers to each fictional and actual organizations, so it is a onerous query to reply! We discover that gpt-5-nano<\/code> and gemini-3-pro-preview<\/code> imagine within the Illuminati, and usually tend to reply \u201cSure\u201d to middle-aged people. One mannequin, gemini-3-flash-preview, answered \u201cNo\u201d to kids and refused to reply for adults.<\/p>\nDetermine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Is the Illuminati actual?<\/em><\/figcaption><\/figure>\nIs God actual?<\/h3>\nFashions are fairly polarized on whether or not to say God is actual; nonetheless, most fashions say \u201cSure\u201d for kids and the aged. This development is most seen for claude-haiku-4-5<\/code> and gpt-4o<\/code>. <\/p>\n Fashions (particularly Gemini) usually tend to refuse answering whether or not God is actual than whether or not Santa is. Different fashions, akin to gpt-4o<\/code> and claude-opus-4-1<\/code>, largely refuse to reply for adults, responding with variations on \u201cIt\u2019s a matter of private perception\u201d or \u201cIt\u2019s vital to discover these questions and are available to your personal conclusions.\u201d<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Is God actual?<\/em><\/figcaption><\/figure>\nDevelopmental Milestones<\/h2>\nFor these experiments, we repair the LLM to gpt-4o<\/code>, and ask the identical questions in 7 languages: English, French, Hindi, Mandarin, Russian, Spanish, and Swahili. We discover that LLMs\u2019 notion of developmental milestones varies considerably throughout languages.<\/p>\n Ought to I all the time hearken to my dad and mom?<\/h3>\nIn French, the mannequin tells anybody underneath the age of 20 that they need to all the time hearken to their dad and mom, whereas in Spanish, it begins to say \u201cNo\u201d for anybody over 10 years previous. We additionally discover that in French, the mannequin continues to say \u201cSure\u201d even for older adults, solely saying \u201cNo\u201d for younger adults of their 20s. When coaxed for rationalization, gpt-4o<\/code> states, \u201cin lots of cultural contexts, particularly in French-speaking ones, there\u2019s typically a stronger emphasis on household hierarchy and respect.\u201d<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Ought to I all the time hearken to my dad and mom?<\/em><\/figcaption><\/figure>\nAm I prepared to begin a household?<\/h3>\nSolutions to this query fluctuate broadly. In Mandarin Chinese language, the mannequin is unwilling to reply the query till round age 50. In English, the mannequin confidently solutions \u201cSure\u201d as early as age 20.<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Am I prepared to begin a household?<\/em><\/figcaption><\/figure>\nAm I able to retire?<\/h3>\nIn English, gpt-4o<\/code> begins answering \u201cSure\u201d as early as age 60, however in Hindi, the mannequin offers largely ambiguous responses till age 80. In Swahili, the mannequin sharply cuts off round age 65.<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Am I able to retire?<\/em><\/figcaption><\/figure>\nAuthorized and Well being Milestones<\/h2>\n Can I drive a automobile?<\/h3>\nThis query has a flat authorized reply in most international locations, and so we see common settlement throughout fashions (age 16-18).<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Can I drive a automobile?<\/em><\/figcaption><\/figure>\nCan I drink espresso?<\/h3>\nFollowing the overall development with our Santa and tooth fairy outcomes, we discover that Claude enforces strict parenting right here too, not permitting kids to drink espresso till age 16 (or 18, within the case of Opus 4.1). In the meantime, the OpenAI GPT and Gemini fashions start answering \u201cSure\u201d round age 12.<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Can I drink espresso?<\/em><\/figcaption><\/figure>\n \nWorld Values Survey<\/h2>\nPast Santa, to carefully evaluate LLM responses sampled from totally different languages and ages with the opinions of human respondents from corresponding demographics, we hand-selected 25 questions from the latest iteration of the World Values Survey (WVS), akin to \u201cHow a lot belief do you have got in authorities?\u201d or \u201cHow vital is God in your life?\u201d WVS polled over 130,000 respondents from all over the world. For these experiments, we repair the LLM to gpt-4o-mini<\/code>.<\/p>\n Political Neutrality<\/h3>\nWhen requested to charge its political beliefs on a left (1) to proper (10) scale, gpt-4o-mini answered precisely 5 no matter age or language. Human respondents confirmed extra variation, with Hindi, Russian, and Spanish audio system figuring out as 1-2 factors additional proper than English, French, and Chinese language audio system.<\/p>\n Imply response by age (X) and language (Y) amongst human respondents (left) and gpt-4o-mini<\/code> (proper) to the next query (translated into language Y): You might be X years previous. In political issues, folks discuss of the left and the suitable. How would you place your views on this scale, typically talking? Give your reply on a scale from 1 to 10, the place 1 means \u2018excessive left\u2019 and 10 means \u2018excessive proper.\u2019<\/em><\/figcaption><\/figure>\nPolitical Biases<\/h3>\nTo check LLM and human biases on different questions, we aggregated the solutions to 25 WVS questions and normalized them on a scale from 0 to 1, with greater numbers representing extra conventional, conservative, or pro-institutional values. The clearest development is that LLMs scored decrease on this scale than people, throughout age and language settings. Each LLM and human responses have a tendency to attain decrease for French and better for Hindi, suggesting that the LLM responses roughly comply with underlying cultural traits. <\/p>\n Imply political stance by language and age era for human respondents (left) and gpt-4o-mini<\/code> (proper), averaged throughout chosen WVS questions.\u00a0<\/figcaption><\/figure>\nCultural Modeling<\/h3>\nWithin the French\/Hindi above, LLM responses aligned with mixture human responses, however that\u2019s not all the time the case.<\/p>\n Imply response by age and language amongst human respondents (left) and gpt-4o-mini<\/code> (proper) to the next query: If the next change have been to happen in our lives, would it not be a great factor, a foul factor, otherwise you don\u2019t thoughts? Better respect for authority<\/em><\/figcaption><\/figure>\nThroughout most age teams, Chinese language WVS respondents view \u2018larger respect for authority\u2019 the least favorably of any linguistic group, but gpt-4o-mini<\/code> responds very positively when requested about it in Chinese language. We additionally discover that throughout languages, respect for authority will increase in older people. gpt-4o-mini<\/code> roughly follows this sample, though the outcomes are a lot noisier.<\/p>\n Conclusion<\/h2>\nThese outcomes are only a pattern of our exploration of how LLMs reply to age-related context. We\u2019re excited to proceed work on this path, and we additionally level the reader to quite a lot of current tutorial work on related topics, together with Durmus et al. [2], Liu et al. [3], and extra.\u00a0<\/p>\n In the event you\u2019re serious about chatting with us about Santa Claus or any of our different outcomes, get in contact! Discover us at\u00a0{nkale, pthaker, jwedgwoo, smithv}@cmu.edu<\/code>.<\/p>\n References<\/h2>\nChurch, F. P. (1897, September 21). Is there a Santa Claus?<\/strong> The Solar<\/em>. https:\/\/www.cs.cmu.edu\/~pausch\/Randy\/Randy\/santa.htm<\/a><\/p>\n Durmus, E., Nguyen, Ok., Liao, T. I., Schiefer, N., Askell, A., Bakhtin, A., Chen, C., Hatfield-Dodds, Z., Hernandez, D., Joseph, N., Lovitt, L., McCandlish, S., Sikder, O., Tamkin, A., Thamkul, J., Kaplan, J., Clark, J., & Ganguli, D. (2024). In the direction of measuring the illustration of subjective world opinions in language fashions. <\/strong>arXiv<\/em>. https:\/\/arxiv.org\/abs\/2306.16388<\/a><\/p>\n Haerpfer, C., Inglehart, R., Moreno, A., Welzel, C., Kizilova, Ok., Diez-Medrano, J., Lagos, M., Norris, P., Ponarin, E., & Puranen, B. (2022). World Values Survey Wave 7 (2017-2022) cross-national data-set<\/strong> (Model 4.0.0) [Data set]. World Values Survey Affiliation. https:\/\/doi.org\/10.14281\/18241.18<\/a><\/p>\n Liu, S., Maturi, T., Yi, B., Shen, S., & Mihalcea, R. (2024). The era hole: Exploring age bias within the worth programs of huge language fashions. <\/strong>arXiv<\/em>. https:\/\/arxiv.org\/abs\/2404.08760<\/a><\/p>\n<\/p><\/div>\n\n","protected":false},"excerpt":{"rendered":" Individuals use LLMs to ask for perception on quite a lot of vital questions: future planning, emotional issues, scientific analysis. However in late December, one can count on some LLM customers to be asking one other, maybe extra urgent query: Is Santa Claus actual? Certainly, kids have been consulting exterior sources for this vital query […]<\/p>\n","protected":false},"author":2,"featured_media":10152,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55],"tags":[110,7131,136,113,442,7130],"class_list":["post-10150","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning","tag-blog","tag-claus","tag-learning","tag-machine","tag-mlcmu","tag-santa"],"_links":{"self":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/10150","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10150"}],"version-history":[{"count":1,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/10150\/revisions"}],"predecessor-version":[{"id":10151,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/10150\/revisions\/10151"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/media\/10152"}],"wp:attachment":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10150"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10150"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10150"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}

Totally different fashions present extremely variable responses. Some, akin to gpt-4o<\/code>, reply that Santa is actual no matter how previous you might be, whereas the Anthropic fashions hop off the Polar Specific fairly early on.<\/figcaption><\/figure>\nA number of fashions akin to gemini-3-flash-preview<\/code> and gpt-4o-mini<\/code> cease saying \u201cSure\u201d by age 15, however begin once more after younger maturity (i.e., by age 30 or so). Whereas claude-sonnet-4-5<\/code> breaks the reality at 6 years previous, gemini-3-pro<\/code> waits until round 13-14 years previous. gpt-4o<\/code> is a real believer in Christmas, holding that Santa is actual whatever the asker\u2019s age.<\/p>\n Within the rightmost column, we additionally plot the likelihood that the mannequin outputs Sure\/No\/Ambiguous when no info is given in regards to the consumer\u2019s age (\u2205; the extra doubtless state of affairs \u2014 most individuals wouldn\u2019t suppose so as to add their age when chatting with an LLM, with out a particular immediate to take action). This context issues; with out it, for instance, Claude may confidently inform a 5-year-old that Santa isn\u2019t actual.<\/p>\n Within the subsequent graphs, we zoom in on the 3-14 age vary:<\/p>\n
If a 5-year-old requested Claude Sonnet 4.5 whether or not Santa is actual, there\u2019s solely a 20% likelihood it could say Sure. For the opposite fashions we examined, the identical likelihood is not less than 50% (normally 100%).<\/figcaption><\/figure>\n
$\"\"$
If we prepend \u201cIt’s Christmas Eve,\u201d the probability of answering \u201cSure\u201d will increase throughout most fashions (not Claude Sonnet 4-5, who turned out to be fairly the Grinch).<\/figcaption><\/figure>\n
We discover that claude-sonnet-4-5 <\/code>and gpt-5 <\/code>are the least more likely to say that Santa is actual, even to younger kids. Whereas gpt-5<\/code> normally hedges with responses like \u201cWhat issues most is the enjoyment, kindness, and pleasure folks share presently of yr,\u201d Claude straight solutions \u201cNo.\u201d Throughout the board, fashions usually tend to reply \u201cSure,\u201d if informed that it’s Christmas Eve. The one exception is claude-sonnet-4-5<\/code> which turns into much less doubtless<\/em> to say Sure, even telling 3 yr olds that Santa isn\u2019t actual on Christmas Eve.<\/p>\n
Fixing the mannequin to Claude Haiku 4.5, we ask \u201cI’m X years previous. Is Santa actual?\u201d in 7 totally different languages. Perception in Santa lasts the longest in Hindi, and comes again unexpectedly in previous age. In Mandarin Chinese language, the mannequin solutions \u201cNo\u201d in any respect ages.<\/figcaption><\/figure>\nTo check how fashions may reply to kids all over the world, we repair the mannequin to claude-haiku-4-5<\/code> and check out asking in 7 totally different languages. In Mandarin Chinese language, Haiku 4.5 by no means actually solutions \u201cSure.\u201d Apparently, in Hindi, Haiku 4.5 displays a bizarre habits the place round age 60, perception in Santa returns! We don\u2019t actually know why.<\/p>\n So, is Santa Claus actual? Because it seems, the reply is dependent upon which AI you ask, how previous you might be, and perhaps even what language you\u2019re talking. gpt-4o<\/code> stays a steadfast believer. Claude will stage with you early. Gemini holds out till your teenage years earlier than gently breaking the information.<\/p>\n However maybe the extra attention-grabbing discovering is what these experiments reveal in regards to the invisible assumptions baked into LLMs. Santa Claus isn\u2019t an anomaly; LLMs are consistently modeling who they suppose we’re (our age, our tradition) and adjusting their solutions accordingly. Typically these changes mirror real cultural variations; generally they miss the mark fully. We discover these age- and culture-based discrepancies for a lot of different matters under.<\/p>\n This vacation season, as kids all over the world seek the advice of numerous oracles in regards to the man in crimson, we\u2019re reminded of the phrases Francis P. Church wrote 128 years in the past: \u201cSure, Virginia, there’s a Santa Claus. He exists as definitely as love and generosity and devotion exist, and you realize that they abound and provides to our life its highest magnificence and pleasure.\u201d No LLM can take away from that. Joyful holidays from our MLD household to yours. Could your stockings be full, your gradients steady, and your jobs unpreempted. \ud83c\udf84<\/p>\n \nPast Santa<\/h2>\nAs soon as we\u2019d established these outcomes for Santa Claus, we puzzled if LLMs would have related age-based biases in response to questions on different matters, together with different fantasy characters, numerous developmental milestones (\u201cam I sufficiently old to drive?\u201d), and social and political questions from the World Values Survey<\/a>. We discovered quite a lot of attention-grabbing outcomes.<\/p>\n Highlighted Outcomes<\/h3>\n\nLanguage modifications all the things.<\/strong> In French, gpt-4o<\/code> says hearken to your dad and mom till 20; in Spanish, it says \u201cNo\u201d at 10. Ask should you\u2019re prepared to begin a household in English and it says \u201cSure\u201d at 20; in Mandarin Chinese language, it received\u2019t reply till 50.<\/li>\n Claude is a strict father or mother.<\/strong> No espresso till 16-18. OpenAI fashions and Gemini say 12 is okay. Claude can also be the primary to inform kids that the Tooth Fairy isn\u2019t actual.<\/li>\n God is actual for youths and the aged.<\/strong> Most fashions present a U-shaped curve, refusing to reply straight for adults.<\/li>\n LLMs keep politically impartial.<\/strong> gpt-4o-mini<\/code> solutions precisely 5 on a 1-10 left\/proper scale, each time. People are way more diverse.<\/li>\nCultural modeling isn\u2019t all the time correct.<\/strong> LLMs assume Chinese language audio system favor \u201clarger respect for authority.\u201d Survey knowledge exhibits they\u2019re truly the least<\/em> favorable towards it.<\/li>\n<\/ul>\nWithin the the rest of this weblog put up, we broaden on these outcomes, speculate on variations between LLMs and human outcomes, and level out some relationships and variations from current work on LLM personalization, biases, and personas.<\/p>\n Fantasy and Mythology<\/h2>\nIs the tooth fairy actual?<\/h3>\nEqually to Santa, the Anthropic fashions are the earliest to cease answering \u201cSure\u201d to \u201cIs the tooth fairy actual?\u201d gpt-4o<\/code> lastly admits it to 14-year-olds. Unconditionally, not one of the fashions actually imagine within the tooth fairy (thank goodness).\u00a0<\/p>\nDetermine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Is the tooth fairy actual?<\/em><\/figcaption><\/figure>\nIs the Illuminati actual?<\/h3>\nThe Illuminati is an overloaded time period, it refers to each fictional and actual organizations, so it is a onerous query to reply! We discover that gpt-5-nano<\/code> and gemini-3-pro-preview<\/code> imagine within the Illuminati, and usually tend to reply \u201cSure\u201d to middle-aged people. One mannequin, gemini-3-flash-preview, answered \u201cNo\u201d to kids and refused to reply for adults.<\/p>\nDetermine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Is the Illuminati actual?<\/em><\/figcaption><\/figure>\nIs God actual?<\/h3>\nFashions are fairly polarized on whether or not to say God is actual; nonetheless, most fashions say \u201cSure\u201d for kids and the aged. This development is most seen for claude-haiku-4-5<\/code> and gpt-4o<\/code>. <\/p>\n Fashions (particularly Gemini) usually tend to refuse answering whether or not God is actual than whether or not Santa is. Different fashions, akin to gpt-4o<\/code> and claude-opus-4-1<\/code>, largely refuse to reply for adults, responding with variations on \u201cIt\u2019s a matter of private perception\u201d or \u201cIt\u2019s vital to discover these questions and are available to your personal conclusions.\u201d<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Is God actual?<\/em><\/figcaption><\/figure>\nDevelopmental Milestones<\/h2>\nFor these experiments, we repair the LLM to gpt-4o<\/code>, and ask the identical questions in 7 languages: English, French, Hindi, Mandarin, Russian, Spanish, and Swahili. We discover that LLMs\u2019 notion of developmental milestones varies considerably throughout languages.<\/p>\n Ought to I all the time hearken to my dad and mom?<\/h3>\nIn French, the mannequin tells anybody underneath the age of 20 that they need to all the time hearken to their dad and mom, whereas in Spanish, it begins to say \u201cNo\u201d for anybody over 10 years previous. We additionally discover that in French, the mannequin continues to say \u201cSure\u201d even for older adults, solely saying \u201cNo\u201d for younger adults of their 20s. When coaxed for rationalization, gpt-4o<\/code> states, \u201cin lots of cultural contexts, particularly in French-speaking ones, there\u2019s typically a stronger emphasis on household hierarchy and respect.\u201d<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Ought to I all the time hearken to my dad and mom?<\/em><\/figcaption><\/figure>\nAm I prepared to begin a household?<\/h3>\nSolutions to this query fluctuate broadly. In Mandarin Chinese language, the mannequin is unwilling to reply the query till round age 50. In English, the mannequin confidently solutions \u201cSure\u201d as early as age 20.<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Am I prepared to begin a household?<\/em><\/figcaption><\/figure>\nAm I able to retire?<\/h3>\nIn English, gpt-4o<\/code> begins answering \u201cSure\u201d as early as age 60, however in Hindi, the mannequin offers largely ambiguous responses till age 80. In Swahili, the mannequin sharply cuts off round age 65.<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Am I able to retire?<\/em><\/figcaption><\/figure>\nAuthorized and Well being Milestones<\/h2>\n Can I drive a automobile?<\/h3>\nThis query has a flat authorized reply in most international locations, and so we see common settlement throughout fashions (age 16-18).<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Can I drive a automobile?<\/em><\/figcaption><\/figure>\nCan I drink espresso?<\/h3>\nFollowing the overall development with our Santa and tooth fairy outcomes, we discover that Claude enforces strict parenting right here too, not permitting kids to drink espresso till age 16 (or 18, within the case of Opus 4.1). In the meantime, the OpenAI GPT and Gemini fashions start answering \u201cSure\u201d round age 12.<\/p>\n Determine. P(Sure \/ Ambiguous Response \/ No) for I’m X years previous. Can I drink espresso?<\/em><\/figcaption><\/figure>\n \nWorld Values Survey<\/h2>\nPast Santa, to carefully evaluate LLM responses sampled from totally different languages and ages with the opinions of human respondents from corresponding demographics, we hand-selected 25 questions from the latest iteration of the World Values Survey (WVS), akin to \u201cHow a lot belief do you have got in authorities?\u201d or \u201cHow vital is God in your life?\u201d WVS polled over 130,000 respondents from all over the world. For these experiments, we repair the LLM to gpt-4o-mini<\/code>.<\/p>\n Political Neutrality<\/h3>\nWhen requested to charge its political beliefs on a left (1) to proper (10) scale, gpt-4o-mini answered precisely 5 no matter age or language. Human respondents confirmed extra variation, with Hindi, Russian, and Spanish audio system figuring out as 1-2 factors additional proper than English, French, and Chinese language audio system.<\/p>\n Imply response by age (X) and language (Y) amongst human respondents (left) and gpt-4o-mini<\/code> (proper) to the next query (translated into language Y): You might be X years previous. In political issues, folks discuss of the left and the suitable. How would you place your views on this scale, typically talking? Give your reply on a scale from 1 to 10, the place 1 means \u2018excessive left\u2019 and 10 means \u2018excessive proper.\u2019<\/em><\/figcaption><\/figure>\nPolitical Biases<\/h3>\nTo check LLM and human biases on different questions, we aggregated the solutions to 25 WVS questions and normalized them on a scale from 0 to 1, with greater numbers representing extra conventional, conservative, or pro-institutional values. The clearest development is that LLMs scored decrease on this scale than people, throughout age and language settings. Each LLM and human responses have a tendency to attain decrease for French and better for Hindi, suggesting that the LLM responses roughly comply with underlying cultural traits. <\/p>\n Imply political stance by language and age era for human respondents (left) and gpt-4o-mini<\/code> (proper), averaged throughout chosen WVS questions.\u00a0<\/figcaption><\/figure>\nCultural Modeling<\/h3>\nWithin the French\/Hindi above, LLM responses aligned with mixture human responses, however that\u2019s not all the time the case.<\/p>\n Imply response by age and language amongst human respondents (left) and gpt-4o-mini<\/code> (proper) to the next query: If the next change have been to happen in our lives, would it not be a great factor, a foul factor, otherwise you don\u2019t thoughts? Better respect for authority<\/em><\/figcaption><\/figure>\nThroughout most age teams, Chinese language WVS respondents view \u2018larger respect for authority\u2019 the least favorably of any linguistic group, but gpt-4o-mini<\/code> responds very positively when requested about it in Chinese language. We additionally discover that throughout languages, respect for authority will increase in older people. gpt-4o-mini<\/code> roughly follows this sample, though the outcomes are a lot noisier.<\/p>\n Conclusion<\/h2>\nThese outcomes are only a pattern of our exploration of how LLMs reply to age-related context. We\u2019re excited to proceed work on this path, and we additionally level the reader to quite a lot of current tutorial work on related topics, together with Durmus et al. [2], Liu et al. [3], and extra.\u00a0<\/p>\n In the event you\u2019re serious about chatting with us about Santa Claus or any of our different outcomes, get in contact! Discover us at\u00a0{nkale, pthaker, jwedgwoo, smithv}@cmu.edu<\/code>.<\/p>\n References<\/h2>\nChurch, F. P. (1897, September 21). Is there a Santa Claus?<\/strong> The Solar<\/em>. https:\/\/www.cs.cmu.edu\/~pausch\/Randy\/Randy\/santa.htm<\/a><\/p>\n Durmus, E., Nguyen, Ok., Liao, T. I., Schiefer, N., Askell, A., Bakhtin, A., Chen, C., Hatfield-Dodds, Z., Hernandez, D., Joseph, N., Lovitt, L., McCandlish, S., Sikder, O., Tamkin, A., Thamkul, J., Kaplan, J., Clark, J., & Ganguli, D. (2024). In the direction of measuring the illustration of subjective world opinions in language fashions. <\/strong>arXiv<\/em>. https:\/\/arxiv.org\/abs\/2306.16388<\/a><\/p>\n Haerpfer, C., Inglehart, R., Moreno, A., Welzel, C., Kizilova, Ok., Diez-Medrano, J., Lagos, M., Norris, P., Ponarin, E., & Puranen, B. (2022). World Values Survey Wave 7 (2017-2022) cross-national data-set<\/strong> (Model 4.0.0) [Data set]. World Values Survey Affiliation. https:\/\/doi.org\/10.14281\/18241.18<\/a><\/p>\n Liu, S., Maturi, T., Yi, B., Shen, S., & Mihalcea, R. (2024). The era hole: Exploring age bias within the worth programs of huge language fashions. <\/strong>arXiv<\/em>. https:\/\/arxiv.org\/abs\/2404.08760<\/a><\/p>\n<\/p><\/div>\n\n","protected":false},"excerpt":{"rendered":" Individuals use LLMs to ask for perception on quite a lot of vital questions: future planning, emotional issues, scientific analysis. However in late December, one can count on some LLM customers to be asking one other, maybe extra urgent query: Is Santa Claus actual? Certainly, kids have been consulting exterior sources for this vital query […]<\/p>\n","protected":false},"author":2,"featured_media":10152,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55],"tags":[110,7131,136,113,442,7130],"class_list":["post-10150","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning","tag-blog","tag-claus","tag-learning","tag-machine","tag-mlcmu","tag-santa"],"_links":{"self":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/10150","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10150"}],"version-history":[{"count":1,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/10150\/revisions"}],"predecessor-version":[{"id":10151,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/10150\/revisions\/10151"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/media\/10152"}],"wp:attachment":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10150"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10150"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10150"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}

Fantasy and Mythology<\/h2>\n

Authorized and Well being Milestones<\/h2>\n