With AI now mainstream in business, a critical question emerges: are we still thinking critically when AI does the heavy lifting? This isn’t just academic — it affects job fulfillment and challenges whether we're employed to do or to think.

Researchers from Carnegie Mellon University and Microsoft Research surveyed knowledge workers who regularly use AI tools, analyzing nearly 1,000 real-world examples. Their findings reveal how our thinking patterns at work are already transforming.

The researchers focused on two key areas:

  • When and how do knowledge workers use critical thinking with generative AI?
  • When and why do these professionals increase or decrease their critical thinking because of these tools?

When Do We Think Critically With AI?

The study uncovered a confidence relationship: the more you trust AI's abilities, the less likely you are to think critically about its outputs. Conversely, professionals with higher self-confidence in their own skills engaged more critically with AI-generated content, despite this requiring more effort.

This creates a potential trap. As AI tools improve and earn our trust, our natural inclination to scrutinize their outputs decreases — precisely when maintaining critical oversight becomes most crucial.

The researchers identified specific motivators and barriers affecting critical thinking with AI. Knowledge workers were motivated to think critically when they wanted to improve work quality, avoid errors or develop professional skills.

However, several barriers prevented critical engagement: awareness barriers by not questioning whether AI was competent for simple tasks, motivation barriers including a lack of time or perceiving critical thinking as outside one’s own job responsibilities and ability barriers due to the inability to verify AI outputs or improve responses.

Lev Tankelevitch, a senior researcher at Microsoft Research and one of the authors of the paper, commented that “our survey-based study suggests that when people view a task as low-stakes, they may not review outputs as critically. However, when the stakes are higher, people naturally engage in more critical evaluation.” Over time, as workers miss opportunities to practice critical thinking in everyday scenarios, this could leave them unprepared for high-stakes situations where these skills become essential.

Is AI Making Critical Thinking Easier or Harder?

For most cognitive activities (knowledge, comprehension, application, analysis, synthesis and evaluation), knowledge workers reported that generative AI had reduced effort.

Beyond just reducing effort, the nature of critical thinking is changing in three fundamental ways:

  1. From information gathering to information verification: AI excels at retrieving and organizing information, but professionals must now invest more energy to ensure information is accurate.
  2. From problem-solving to response integration: While AI efficiently generates solutions, knowledge workers must adapt these outputs to specific contexts.
  3. From task execution to task stewardship: Knowledge workers are shifting from performing tasks themselves to guiding and overseeing AI completion of these tasks.

Future Job Design

These shifts in critical thinking patterns will profoundly impact the future of work in several ways.

First, organizational structures will likely evolve to emphasize oversight roles. We will continue to see new positions focused specifically on AI prompt engineering, output verification and quality control.

Second, performance evaluation metrics will need recalibration. Traditional metrics often measure task execution speed and quality, but in an AI-augmented workplace, the ability to effectively direct and evaluate AI outputs may become more valuable than personal execution capabilities.

Third, workplaces will need to address the issue of automating routine cognitive tasks which inadvertently erode the everyday practice opportunities that develop critical thinking skills. Just as calculators changed how we approach mental math, AI tools may fundamentally alter how we develop analysis and evaluation skills. This creates an irony of automation problem where the greater the automation, the greater the need for oversight and yet the lack of experience to provide this oversight.

Forward-thinking organizations will design deliberate practice opportunities for critical thinking including incorporating verification steps into workflows to maintain critical engagement.

Irritating as it may be, future AI interfaces could intentionally prompt critical reflection rather than encouraging passive acceptance of outputs. This may give rise to cognitive forcing functions that require users to actively engage with AI responses before proceeding.

Future Skills

The skills most valued in knowledge workers are evolving as a result. Domain expertise remains crucial — you can’t effectively verify AI outputs without it — but this expertise now pairs with new competencies in AI direction, evaluation and integration.

The transformation of critical thinking in the AI era doesn’t signal the end of this crucial skill, but rather its evolution. As knowledge work increasingly involves collaboration with artificial intelligence, our capacity for thoughtful oversight, verification and integration will define workplace success.

Tankelevitch adds, “Across all of our research, there is a common thread: AI works best as a thought partner, complementing the work people do. When AI challenges us, it doesn’t just boost productivity; it drives better decisions and stronger outcomes.”

Those who thrive won't be those who most enthusiastically embrace or reject AI, but those who develop a balanced approach that leverages AI capabilities while maintaining the critical thinking skills that remain uniquely human.