Critical Evaluation of Journals and Books (both print and online)

It is always important to critically evaluate information that you are using. Sometimes this is easier to do than others. Many printed sources go through an editorial review process which means that either one person or several people agree that the information is credible before allowing it to be published. Newspaper articles, magazine articles, and scholarly journal articles all go through this process.

Just because something is published, doesn't mean you should trust it. Consider the supermarket tabloids like the Weekly World News. Certainly, that information source is suspect! Here are some other scenarios that should cause you to think more critically about what you read:

Magazines accept advertising. Universities accept industry donations. In theory, researchers and journalists are not supposed to take this into account when conducting research or writing an article criticizing a company or a product, but it is something that you as a reader should consider as a possibility.
Newspapers, magazines, and television stations are now often owned by large entertainment companies (for example, Time Warner owns Time Magazine, CNN, HBO, New Line Cinema, AOL, and Warner Bros., Sports Illustrated, and Warner Books). Think about that the next time you read a review of a new album, book, movie, or television show.
Even scholarly journals are not exempt. In the last 5 years, peer review scandals and hoaxes have led to the retraction of peer-reviewed journal articles in both the sciences and the social sciences. Most journals do not publicize article retractions so that the public is mostly unaware of the discredited work. The Retraction Watch blog tracks retracted journal articles.

Just because you find information on the Web does not mean it is automatically false NOR does it mean you can automatically trust it. It needs to be critically evaluated just like any printed source, and in some cases even MORE critically evaluated since many websites have no editor or reviewers. There are some websites which are entirely fictitious. Using reviewed sources, comparing the information you find in one source to other sources on the same topic, and corroborating information are three useful strategies for ensuring the quality of your information.

Below are some questions that can guide you through the process of thinking critically about the information source you are considering using. Keep in mind that this process may take some time. It isn't necessary to answer all the questions, but it is important that you think through them before using ANY source of information from a book, to a website to an "expert" whom you interview.

Authority

To evaluate authority:

Questions to Ask	Finding the Answers
Who is the author?	Can you identify an author for the work? Most common places to find authors' names listed: Title page (book or report) Title information at top of first page (articles, book chapters) End of the article (encyclopedias) Top or bottom of page (web pages)
What are the author's credentials? Relevant university degree Institutional affiliation (where does he or she work?) Relevant field or employment experience Past writings	Examine the item for information about the author Look in biographical sources Search the web for the author's home page If the author is affiliated with a university, search for a profile on the university's website Search scholarly databases and library catalogues for other works by the author
What is the author's reputation among his/her peers? Cited in articles, books or bibliographies on the topic Mentioned in your textbook or by your professor	Use citation indexes to find articles citing your author Google Scholar results indicate how many times an article was cited.
Who is the publisher? Commercial, trade, institutional, other Known for quality and/or scholarly publications Basic values or goals Specialization Editorial board Blind review process	Search the web to learn what other sites say about the publisher. Information about larger publishing companies may also be available in the Mount Library's Business databases. Search the web for the publisher's web site Look for editorial guidelines or author instructions in journals or on the publisher's web site Information is also available in directories, though they can be difficult to access e.g. Writer's Market Literary Marketplace Directory of Corporate Affiliations
Who is the funder? government, NGO, institutional mandate and reputation of agency review process size of grant	Funders should be acknowledged in article Visit their web site and review granting guidelines
Is the author associated with a reputable institution or organization? Organizational mission Basic values or goals National or international Membership	Search the web to learn what other sites say about the organization. Search the web for the organization's web site.

Objectivity

To evaluate objectivity

Questions to Ask	Finding the Answers
Does the author state the goals for this publication? Inform, explain, educate Advocate Persuade or dissuade Sell a product or service Serve as a soapbox	Read the foreword, preface, abstract and/or introduction Look for reviews of the work in article indexes.
Does the author exhibit a particular bias? What assumptions does the author make? Commitment to a point of view Acknowledgement of bias Presentation of facts and arguments for both sides of a controversial issue Language free of emotion-arousing words and bias. How does author's point of view or claims compare to others on same topic?	Read the abstract and/or introduction Examine the work for Inflammatory language, images or graphic styles (boldface, italics to make a point or indicate sarcasm) Propaganda Author's arguments or supporting facts Author's conclusions Bibliography that includes multiple points of view Search for articles or books or websites on same topic
Is the viewpoint of the author's affiliation reflected in the message or content? Organization's point of view on the topic being discussed Organization's mission and activities Advertising is clearly labeled Benefits to the organization	Search the web for the organization's web site Look in directories
Does the information appear to be valid and well-researched? Reasonable assumptions and conclusions Arguments and conclusions supported by evidence that is verifiable Opposing points of view addressed Opinions not disguised as facts Authoritative sources cited.	Verify facts and statistics with other reliable sources

Quality

To evaluate quality

Questions to Ask	Finding the Answers
Is the information well-organized? Logical structure Main points clearly presented Main ideas unified by overarching idea Text flows well (not choppy or stilted) Author's argument is not repetitive	Look at the headings to indicate structure Book reviews (indexed in many article databases) Look for agreement among reviews Internet Scout Report
Has author used good grammar? Are there spelling or typographical errors?	Read carefully for errors
Are the graphics (images, tables, charts, diagrams) appropriate and clearly presented? Clearly labeled Descriptive title Understandable without explanatory text	Consider other ways to present the information

Coverage

To evaluate coverage & corroboration

Questions to Ask	Finding the Answers
Does the work update other sources?	Compare publication dates and content to other sources you have found
Does it corroborate other materials you have read, or add new information?	You should seek out multiple points of view and include a diversity of sources and ideas
If there are quotations from other sources or other "pieces" of information, are they taken out of context?	Verify imporant quotes or other "pieces" of information by examining them in their original source documents. For example, if the quote is taken from a magazine or book, find that item and read the quote in the context of the whole section or paragraph.
Have you found enough information to support your arguments?	Look for gaps in your arguments and evidence Facts Statistics Evidence

Currency

To evaluate currency

Questions to Ask	Finding the Answers
When was it published?	Look for a publication or copyright date on the Title page (books, journals) Reverse of the title page (books) Cover (journals, magazines, newspapers) Table of contents (journals, magazines) Bottom of page (web sites) Dates on web pages may indicate When the page was created When the page was published on the web When the page was last revised The copyright date for the entire website, rather than a date related to the content of the page.
Is your topic one that requires current information?	Topic areas requiring the most up-to-date information may include Science Medicine Current events
Has this source been revised, updated, or expanded in a later edition?	Search catalogs and other databases for more recent editions Novanet Library Search Databases featuring book reviews Amazon.ca

Questions to Ask

Finding the Answers

When was it published?

Look for a publication or copyright date on the

Title page (books, journals)
Reverse of the title page (books)
Cover (journals, magazines, newspapers)
Table of contents (journals, magazines)
Bottom of page (web sites)

Dates on web pages may indicate

When the page was created
When the page was published on the web
When the page was last revised
The copyright date for the entire website, rather than a date related to the content of the page.

Is your topic one that requires current information?

Topic areas requiring the most up-to-date information may include

Science
Medicine
Current events

Has this source been revised, updated, or expanded in a later edition?

Search catalogs and other databases for more recent editions

Relevance

To evaluate relevance

Questions to Ask	Finding the Answers
Does the work address your research question or meet the requirements of your assignment?	Review your research question and/or assignment
Is the content appropriate for your research topic or assignment? Scholarly vs. popular Fact vs. opinion Format/medium (e.g. book, journal, website, etc.) Subject coverage Language Time period Geographical area Audience Primary vs. Secondary	Check the table of contents or scan the subheadings Read the preface, abstract, introduction, and/or conclusion Look for footnotes or endnotes and/or a bibliography Look for reviews Book reviews Internet Scout Report

Acknowledgement

The staff of the Mount Saint Vincent University Library & Archives wish to acknowledge and thank colleagues at the University of Oregon and the University of Louisville for their kind permission to copy the content from their critical evaluation guides.

Evaluating Sources

A tutorial describing how to evaluate sources by Western Libraries. Please contact a MSVU Librarian for more assistance.

How to spot bad science

12 steps to spotting bad science
Misinterpreted results, conflict of interest, sample size too small, no control group, unreplicable results, and more!

Which research to trust?

Evaluating health research
The CDC article: “How do you know which health care effectiveness research you can trust? A guide to study design for the perplexed” has 5 case studies to help people better understand health research.
Predatory Publishers
University of Saskatchewan's guide to assist researchers in recognizing and avoiding questionable or disreputable publishers and journals (commonly called "predatory publishers.")

Help with Library Research