What I've Learned: Text Mining Journey

Different Worlds of Communication

Perhaps the most striking discovery was how dramatically different the language is between various climate discourse communities. Scientists, activists, policymakers, and skeptics aren't merely disagreeing—they're often speaking entirely different languages. Terms that carry significant meaning in one community may be absent or have different connotations in another.

The Power of Perspective

My analysis revealed how the same climate topics can be framed in radically different ways that evoke entirely different emotional responses. When environmental concerns are framed as economic opportunities versus regulatory burdens, it completely changes public reception—even when discussing identical issues.

This awareness of framing has changed how I approach both consuming and creating content across all domains, making me more conscious of the subtle ways information presentation shapes our understanding.

The importance of Discourse

Text mining highlighted how easily we can become trapped in communication bubbles. The context that surrounds us shapes the perception of our enviornments and truths.The distinct vocabulary and narrative patterns in different communities create self-reinforcing information environments (sometimes misinforming). Breaking through these requires conscious effort and translation between different ways of speaking about issues, or open communication.

I've become more aware of my own information bubble and now actively seek diverse perspectives, even on topics where I hold strong views.

The Foundation of Quality: Data Labeling Matters

One of the most profound lessons from this project was discovering just how crucial proper data labeling and quality are to the entire machine learning process. When I carefully labeled climate texts with lexicon-based approaches and manually reviewed examples, my models performed dramatically better than with automated or generic approaches. This also be a double egded sword in the sense that if I am programmatically labeling something and then letting a model to train on that data, there is a high chance that the model will just cling to the rules that I used to label.

The old saying "garbage in, garbage out" took on new meaning—no matter how sophisticated my neural networks or algorithms became, they could never overcome fundamental data quality issues. Investing time in creating thoughtful, balanced(resampling), and representative training data yielded insights that would have remained hidden otherwise.

Beyond technical accuracy, I learned that labeling decisions themselves embed values and perspectives that shape all downstream insights. The choices we make about what constitutes "positive" or "negative" sentiment in climate discourse, for example, aren't just technical decisions—they're interpretive acts that influence what patterns our models can discover.