Skip to Main Content
Guides

Source Evaluation guide

Find tools, tips, and tricks for evaluating print sources, websites, and other online resources.

Introduction

Evaluating source material from Generative AI

There are no frameworks aimed explicitly at assessing information from Gen AI like ChatGPT, so we asked ChatGPT what it thought people should do when determining whether the information provided is credible or not. This list has many parallels to established frameworks, and these tips can be applied to just about any source.

Below is a direct quote from ChatGPT.

Image source: pixabay

From ChatGPT

Evaluating information from an AI tool like ChatGPT involves several key steps to ensure reliability, accuracy, and relevance. Here are some recommended source evaluation methods:

1. Cross-Verification

  • Check Multiple Sources: Verify the information provided by ChatGPT against multiple credible sources. This includes academic papers, official reports, reputable news outlets, and expert opinions.
  • Consistency: Look for consistency across different sources. If multiple reliable sources confirm the same information, it is more likely to be accurate.

2. Authority and Credibility

  • Expertise: Assess the expertise of the sources referenced by ChatGPT. Information from recognized experts or institutions is generally more reliable.
  • Reputation: Consider the reputation of the sources. Established and well-regarded publications and organizations are more likely to provide trustworthy information.

3. Accuracy and Evidence

  • Citations: Check if ChatGPT provides citations or references for the information. Reliable information often comes with clear sources or references that can be checked.
  • Evidence-Based: Ensure that the information is based on empirical evidence or well-supported data rather than opinions or unverified claims.

4. Bias and Objectivity

  • Perspective: Be aware of potential biases in the sources referenced. Consider different perspectives and look for balanced viewpoints.
  • Objective Analysis: Prefer sources that provide objective analysis and avoid those with obvious bias or agenda-driven content.

5. Timeliness

  • Current Information: Check the date of the information provided. Ensure that it is up-to-date and relevant to the current context.

6. Relevance

  • Context: Ensure that the information is relevant to your specific research question or topic.
  • Specificity: Look for information that is specific and detailed enough to be useful for your needs.

7. Critical Thinking

  • Question Assumptions: Critically evaluate the assumptions underlying the information. Are there any logical fallacies or unsupported conclusions?
  • Analyze Arguments: Consider the strength and validity of the arguments presented. Are they well-reasoned and supported by evidence?

8. Feedback and Peer Review

  • Consult Peers: Discuss the information with peers or experts in the field to get their opinions and feedback.
  • Review: If possible, have the information reviewed by someone knowledgeable to ensure its accuracy and reliability.

 

OpenAI. (2024, July 16). [Methods for evaluating information from AI tools]. ChatGPT. https://www.openai.com/chatgpt