, I submitted my MS thesis on Emotion Recognition in Dialog (ERC). The mannequin,\u00a0EmoNet<\/strong>, achieved a Weighted F1 of\u00a039.18 on EmoryNLP<\/strong>\u00a0\u2014 aggressive with the general public PapersWithCode leaderboard on the time, sitting between TUCORE-GCN_RoBERTa (39.24) and S+PAGE (39.14), and bettering over my chosen baseline, CoMPM, by\u00a0+1.81 F1<\/strong>.<\/p>\n Two years later, I returned to take a look at the place the sphere is now. The leaderboard is unrecognizable. The highest entries are not encoder-only fashions with intelligent consideration heads \u2014 they\u2019re\u00a0LLaMA-2\u20137B-based methods with LoRA fine-tuning and retrieval-augmented prompting<\/strong>: InstructERC, CKERC, BiosERC, LaERC-S. The strategies are totally different. The compute is totally different. The mindset is totally different.<\/p>\n And but \u2014 once I learn these new papers rigorously,\u00a0the core concepts I proposed in EmoNet present up inside them, simply applied at a distinct layer of the stack.<\/strong>\u00a0That is the story of what I constructed, the place it positioned, and what I\u2019d construct now if I have been beginning over.<\/p>\n Emotion Recognition in Dialog is the duty of assigning an emotion label to every utterance in a multi-turn dialogue. It\u2019s distinct from sentiment evaluation on remoted sentences in a single vital approach:\u00a0the emotion of an utterance is formed by what got here earlier than it, and by who’s talking.<\/strong><\/p>\n Take into account this alternate from the EmoryNLP dataset (sourced from the TV present\u00a0Pals<\/em>):<\/p>\n Monica:<\/em><\/strong>\u00a0Wendy, we had a deal! Yeah, you promised! Wendy! Wendy! Wendy! \u00a0 [Mad]<\/em><\/p>\n Rachel:<\/em><\/strong>\u00a0Who was that? \u00a0 [Neutral]<\/em><\/p>\n Monica:<\/em><\/strong>\u00a0Wendy bailed. I’ve no waitress. \u00a0 [Mad]<\/em><\/p>\n In isolation, \u201cWho was that?\u201d is emotionally impartial. The label\u00a0Impartial<\/em>\u00a0is simply significant\u00a0in context<\/strong>\u00a0\u2014 it sits between two indignant utterances from a distinct speaker and ERC fashions should seize this conversational dynamic.<\/p>\n There\u2019s a second wrinkle:\u00a0multimodal data is lacking.<\/strong>\u00a0In actual human dialog, tone of voice, facial expressions, and physique language carry an unlimited share of emotional sign. Textual content-only ERC strips all of that away. The identical phrases \u2014 \u201cOh, nice.\u201d \u2014 will be honest or sarcastic, and the textual content alone typically can\u2019t inform you which.<\/p>\n This data loss is the central problem. You need to extract emotion from a noisier sign than the human-grade benchmark.<\/p>\n After I began my thesis in late 2023, the EmoryNLP leaderboard was dominated by transformer-based architectures with numerous intelligent modifications. A fast tour:<\/p>\n \u2013\u00a0KET<\/strong>\u00a0(Zhong et al., 2019) \u2014 knowledge-enriched transformer with affective graph consideration, the primary paper to carry transformers to ERC.<\/p>\n \u2013\u00a0DialogueGCN<\/strong>\u00a0(Ghosal et al., 2019) \u2014 graph convolutional community that transformed dialogues into node-classification issues.<\/p>\n \u2013\u00a0RGAT<\/strong>\u00a0(Ishiwatari et al., 2020) \u2014 relation-aware graph consideration with relational place encoding for speaker dependencies.<\/p>\n \u2013\u00a0DialogXL<\/strong>\u00a0(Shen et al., 2020) \u2014 tailored XLNet with utterance recurrence and dialogue self-attention.<\/p>\n \u2013\u00a0HiTrans<\/strong>\u00a0(Li et al., 2020) \u2014 hierarchical transformer with pairwise utterance speaker verification as auxiliary process.<\/p>\n \u2013\u00a0TUCORE-GCN<\/strong>\u00a0(Lee & Choi, 2021) \u2014 heterogeneous dialogue graph with speaker-aware BERT.<\/p>\n \u2013\u00a0CoMPM<\/strong>\u00a0(Lee & Lee, 2021) \u2014 mixed dialogue context with pre-trained reminiscence monitoring for the speaker.<\/p>\n I selected\u00a0CoMPM as my base<\/strong>\u00a0for 2 causes. First, it explicitly modeled the speaker\u2019s pre-trained reminiscence as a separate module \u2014 which mapped to my instinct that\u00a0who<\/em>\u00a0is talking issues as a lot as\u00a0what<\/em>\u00a0they\u2019re saying. Second, its structure was modular sufficient to increase with out rewriting from scratch. The CoMPM paper confirmed that including pre-trained reminiscence to the context mannequin gave a measurable enhance \u2014 however their speaker id was nonetheless\u00a0native to every dialogue<\/strong>. The second a brand new dialog started, all the things the mannequin had discovered a few speaker was discarded.<\/p>\n That appeared like an issue price fixing.<\/p>\n The issue.<\/strong>\u00a0In CoMPM and most prior work, speaker IDs are scoped to a single dialogue.\u00a0Speaker A<\/em>\u00a0in scene 1 has no relationship to\u00a0Speaker A<\/em>\u00a0in scene 14, even after they\u2019re the identical particular person. Therefore, each dialogue begins chilly.<\/p>\n The instinct.<\/strong>\u00a0Folks have attribute emotional patterns. Monica will get indignant about particular issues; Phoebe is reliably cheerful; Ross has predictable bouts of insecurity. If a mannequin can carry details about\u00a0this particular speaker<\/em>\u00a0throughout dialogues, it ought to have the ability to make better-calibrated predictions when that speaker reappears.<\/p>\n The implementation.<\/strong>\u00a0Every distinctive speaker in the complete dataset will get a secure, dataset-wide ID. The primary time\u00a0Monica Geller<\/em>\u00a0seems, she\u2019s assigned an ID \u2014 say, ID 7 \u2014 that stays along with her. Each subsequent look \u2014 throughout episodes, seasons, scenes \u2014 she stays ID 7. The mannequin can now study speaker-specific patterns that persist.<\/p>\n This sounds apparent on reflection. In 2024 it was not how the leaderboard fashions labored.<\/p>\n The issue.<\/strong>\u00a0International Speaker Identification alone is only a label. To make it helpful, the mannequin must\u00a0do one thing<\/em>\u00a0with the speaker\u2019s accrued historical past. How do you give a transformer entry to \u201call the things Monica has ever stated on this dataset,\u201d with out blowing out the context window or making coaching intractable?<\/p>\n The instinct.<\/strong>\u00a0Recurrence. A GRU is a pure match for sequentially compressing a speaker\u2019s historic utterances right into a single fixed-size illustration. Latest utterances contribute extra; older ones step by step dilute. A configurable\u00a0sliding window<\/strong>\u00a0bounds the GRU\u2019s enter \u2014 say, the final N utterances by this speaker \u2014 conserving compute and reminiscence predictable.<\/p>\n The implementation.<\/strong>\u00a0Every utterance is independently encoded by a pre-trained RoBERTa spine. The ensuing embeddings stream via a unidirectional GRU. The GRU\u2019s remaining hidden state \u2014 name it `kt` \u2014 represents the speaker\u2019s behavioral sample on the present second. That is projected into the identical dimension because the dialogue context output and added in. The mixed sign feeds the ultimate classifier.<\/p>\n The structure is structurally much like CoMPM\u2019s pre-trained reminiscence module, however with two key variations: the speaker-history pool is\u00a0world<\/strong>\u00a0(not native to the present dialogue), and the GRU explicitly fashions temporal decay.<\/p>\n The issue.<\/strong>\u00a0EmoryNLP is imbalanced \u2014\u00a0Impartial\u00a0<\/em>outnumbers\u00a0Unhappy<\/em>\u00a0by roughly 4.5:1. Most papers deal with this with knowledge augmentation or under-sampling. However conversational knowledge is\u00a0sequential<\/strong>: dropping or duplicating utterances distorts the pure emotional stream, which is precisely the sign the mannequin is making an attempt to study from.<\/p>\n The instinct.<\/strong>\u00a0In case you can\u2019t safely change the information, change the loss. Weight uncommon lessons larger so a single misclassification of\u00a0Unhappy<\/em>\u00a0prices the mannequin greater than a single misclassification of\u00a0Impartial<\/em>.<\/p>\n The implementation.<\/strong>\u00a0Cross-entropy with per-class weights derived from inverse class frequency, then normalized. Nothing unique \u2014 however with the conversational-sequence argument as the specific motivation, this turns into a principled alternative slightly than an arbitrary one.<\/p>\n Right here\u2019s the ablation desk from the thesis:<\/p>\n The outcome that stunned me \u2014 and that I feel is probably the most trustworthy a part of this work \u2014 is the second row.\u00a0Including International Speaker ID alone made the mannequin considerably worse<\/strong>\u00a0(F1 dropped from 37.85 to 29.43). That seemed like a failure at first.<\/p>\n Nevertheless it wasn\u2019t. The International Speaker Identification is a\u00a0functionality<\/em>\u00a0\u2014 it offers the mannequin the power to study long-range speaker patterns. By itself, that functionality creates a representational burden the remainder of the mannequin couldn\u2019t soak up. Solely as soon as the\u00a0Speaker Behaviour module<\/strong>\u00a0was added \u2014 giving the mannequin a structured strategy to\u00a0use<\/em>\u00a0the worldwide identities \u2014 did the contribution floor. By the ultimate configuration, EmoNet had recovered and surpassed the CoMPM baseline by 1.81 F1.<\/p>\n That is the lesson I took from the ablation:\u00a0a characteristic isn\u2019t helpful in isolation; it\u2019s helpful together with the equipment that consumes it.<\/strong>\u00a0Analysis papers that report \u201cthis addition gave us +X%\u201d typically cover ablation rows the place the addition alone made issues worse. I selected to maintain that row in.<\/p>\n The total mannequin dealt with\u00a0Impartial<\/em>,\u00a0Pleasure<\/em>, and\u00a0Scared<\/em>\u00a0effectively.\u00a0Highly effective<\/em>\u00a0remained the toughest class\u200a\u2014\u200apartly as a result of it\u2019s uncommon, and partly as a result of\u00a0Highly effective<\/em>\u00a0and\u00a0Pleasure<\/em>\u00a0are practically indistinguishable in textual dialog with out acoustic cues. It is a multimodal drawback masquerading as a textual content drawback.<\/p>\n Two years on, the EmoryNLP leaderboard appears fully totally different. The main methods now are:<\/p>\n \u2013\u00a0InstructERC<\/strong>\u00a0(Lei et al., 2023) \u2014 reformulates ERC as a generative LLM process. It makes use of retrieval-augmented instruction templates and auxiliary duties resembling speaker identification and emotion prediction to higher mannequin dialogue roles and emotional dynamics.<\/p>\n \u2013\u00a0CKERC<\/strong>\u00a0(Fu, 2024) \u2014 introduces commonsense-enhanced ERC. For every utterance, an LLM generates commonsense annotations about speaker intention and sure listener response, offering implicit social and emotional reasoning past specific dialogue context.<\/p>\n \u2013\u00a0BiosERC<\/strong>\u00a0(Xue et al., 2024) \u2014 injects LLM-derived speaker biographical data into the ERC course of, permitting the mannequin to motive not solely over utterance context but in addition over speaker-specific traits.<\/p>\n \u2013\u00a0LaERC-S<\/strong>\u00a0(Fu et al., 2025) \u2014 two-stage instruction tuning. Stage 1: equip the LLM with\u00a0speaker-specific traits<\/strong>. Stage 2: use these traits throughout the ERC process itself.<\/p>\n Take a look at these final two rigorously.<\/p>\n BiosERC\u2019s speaker biographical data<\/strong>\u00a0is, in spirit, International Speaker Identification scaled up \u2014 as an alternative of an integer ID, it\u2019s a textual profile the LLM can attend to.\u00a0LaERC-S\u2019s speaker traits<\/strong>\u00a0are, in spirit, the Speaker Behaviour module \u2014 historic speaker patterns made out there to the mannequin \u2014 however folded into instruction tuning slightly than applied as a separate GRU.<\/p>\n The architectural intuitions held up. The implementation layer modified.<\/p>\n That is the half I discover genuinely attention-grabbing. After I was engaged on EmoNet in 2024, I used to be considering contained in the encoder-only-transformer paradigm: \u201chow do I add one other module to the structure?\u201d The 2024\u20132025 papers assume contained in the LLM paradigm: \u201chow do I encode this concept into instruction tuning or retrieval context?\u201d The concepts are comparable; the leverage factors are totally different.<\/p>\n If I have been to rebuild EmoNet right now<\/strong>, I might not begin from RoBERTa-large. I might begin from a small open-source LLM \u2014 LLaMA-3.2\u20133B, Qwen-2.5\u20133B, or Phi-3.5 \u2014 and use\u00a0LoRA<\/strong>\u00a0to fine-tune it on EmoryNLP, following the InstructERC household of approaches. The International Speaker Identification turns into a textual speaker biography retrieved from a vector retailer. The Speaker Behaviour module turns into a few-shot immediate with the speaker\u2019s most up-to-date emotional historical past. The Weighted Loss survives nearly unchanged \u2014 class imbalance doesn\u2019t care what mannequin you\u2019re utilizing.<\/p>\n The structure diagram would look fully totally different. The conceptual debt to the 2024 thesis could be seen should you knew the place to look.<\/p>\n It taught me that analysis debt has an extended half-life than I anticipated \u2014 concepts survive paradigm shifts even when their implementations don\u2019t.<\/em><\/p>\n EmoNet is now publicly archived below\u00a0DOI 10.5281\/zenodo.20048006<\/a>\u00a0with the total thesis, protection slides, and PyTorch implementation on\u00a0GitHub<\/a>. I\u2019m at present engaged on the modernized port \u2014 a LoRA-fine-tuned LLM with retrieval-based speaker context \u2014 as a follow-up undertaking that I\u2019ll write about quickly.<\/p>\n In case you\u2019re engaged on conversational AI, utilized NLP, or LLM fine-tuning, I\u2019d have an interest to listen to what you\u2019re constructing.<\/p>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":" , I submitted my MS thesis on Emotion Recognition in Dialog (ERC). The mannequin,\u00a0EmoNet, achieved a Weighted F1 of\u00a039.18 on EmoryNLP\u00a0\u2014 aggressive with the general public PapersWithCode leaderboard on the time, sitting between TUCORE-GCN_RoBERTa (39.24) and S+PAGE (39.14), and bettering over my chosen baseline, CoMPM, by\u00a0+1.81 F1. Two years later, I returned to take a […]<\/p>\n","protected":false},"author":2,"featured_media":15217,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55],"tags":[73,5128,9248,5989,2359,9249,7101],"class_list":["post-15215","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning","tag-build","tag-differently","tag-emonet","tag-emotion","tag-recognition","tag-speakeraware","tag-transformers"],"_links":{"self":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/15215","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=15215"}],"version-history":[{"count":1,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/15215\/revisions"}],"predecessor-version":[{"id":15216,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/15215\/revisions\/15216"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/media\/15217"}],"wp:attachment":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=15215"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=15215"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=15215"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nWhat ERC is, and why text-only is tough<\/h2>\n
\nThe 2024 panorama<\/h2>\n
\nThree contributions, with instinct<\/h2>\n
1. International Speaker Identification<\/h3>\n
2. Speaker Behaviour Module<\/h3>\n
![\"\"](\"https:\/\/contributor.insightmediagroup.io\/wp-content\/uploads\/2026\/05\/image-370.png\")
3. Weighted Cross-Entropy Loss<\/h3>\n
\nOutcomes: what labored, and what stunned me<\/h2>\n
![\"\"](\"https:\/\/contributor.insightmediagroup.io\/wp-content\/uploads\/2026\/05\/image-374.png\")
<\/figure>\n![\"\"](\"https:\/\/contributor.insightmediagroup.io\/wp-content\/uploads\/2026\/05\/image-376.png\")
<\/figure>\n
\nReflection (2026): the sphere moved, and so ought to we<\/h2>\n
\nThe place this leaves me<\/h2>\n