On the Internet As approved by the IEEE-USA The Internet and Web-based technologies provide users with a powerful tool to access health information. IEEE-USA recognizes the importance of the quality of health information on the Internet and endorses the Health Summit Working Group (HSWG) policy paper, "Criteria for Assessing the Quality of Health Information on the Internet," (see attached appendix) as a minimum set of criteria for assessing the quality of health information on the Internet. The IEEE-USA recommends the development of technology-based tools that support the assessment and development of quality health information on the Internet. These tools should be based on the minimum set of criteria. These criteria, based upon the seven criteria developed by HSWG, are abbreviated as follows:
This statement was developed by the IEEE-USA Medical Technology Policy Committee and represents the considered judgment of a group of U.S. IEEE members with expertise in the subject field. IEEE-USA promotes the career and technology-policy interests of the nearly 225,000 electrical, electronics and computer engineers who are U.S. members of the IEEE. BACKGROUND The Issue: The Internet can be a valuable resource for users seeking health information. The quality of this information is critically important as it could potentially affect health outcomes for millions. Yet the quality of health information on the Internet is extremely variable and difficult to assess. As health information proliferates on the Internet, there is thus, a growing need for objective criteria that can be used to evaluate the quality of information. In 1999, the Federal Trade Commission (FTC) announced a new project for consumer and business education using Web sites to warn them of fraudulent advertising for health products on the Internet. The Food and Drug Administration (FDA) is also taking action against Internet site operators if claims made for medical products violate labeling laws. However, there are far too many sites for the federal government to police effectively. In 1998, more than 22 million Americans used their computers to seek medical information, making health concerns the sixth most common reason for using the Internet. The number of people going online for this purpose is growing by 70 percent a year and there are thousands of sites promoting health products on the Internet. New, Internet-based educational tools will have to be developed in the private sector to complement government information quality tools that provide an ability to assess the quality of health information on the Internet. The Health Summit Working Group: In November 1996, the Health Information Technology Institute of Mitretek Systems (which is a non-profit, 501(c)(3) organization) convened the Health Summit Working Group (HSWG), a diverse group of stakeholders including consumers, health care providers, librarians, developers, academicians and government. The Working Group addressed the growing need for objective, widely acceptable criteria that can be used to evaluate the quality of information on the Internet. It has developed a policy paper that presents a set of seven criteria for use in evaluating the quality of health information provided on the Internet. The Institute
of Electrical and Electronics Engineers - United States of
America | Top of Page | Position Statements | Policy Forum | IEEE-USA | APPENDIX Criteria for
Assessing the Quality POLICY PAPER Health Summit Working Group Supported in part by the Agency
for http://hitiweb.mitretek.org/hswg This policy paper presents a set of seven criteria developed for use in evaluating the quality of health information provided on the Internet. It is hoped that organizations and individuals will become involved in this important effort at some level, whether it be supporting and/or endorsing the policy paper, adopting the use of the set of criteria, joining the Health Summit Working Group, or educating others on this topic. Background As health information proliferates on the Internet, there is a growing need for objective, reproducible, widely accepted criteria that can be used to evaluate the quality of the information. To address this need, the Health Information Technology Institute of Mitretek Systems convened a Health Summit Working Group. Participants included representatives of the general public, as well as health-care providers, medical librarians and related information resources professionals, and website developers affiliated with numerous organizations in the health-care and information communities. The group held three Health Summit Meetings over a period of 18 months (November 1996-May 1998). The first and a portion of the second meeting were focused on creating and refining a set of criteria for assessing the quality of health information on the Internet. Broad input and outside review of these criteria were solicited at a number of medical and scientific meetings, and an interim white paper was posted on the Web for comment. The revised set of criteria that resulted from that process is presented in this document. The second and third Health Summit Meetings centered on implementing these criteria into a Web-based, interactive tool. The tool development, the testing process used to validate the tool, and the test results are beyond the scope of this paper, but will be presented in a subsequent paper. The Problem The Internet can be a valuable resource for users seeking health information. The quality of this information is critically important as it could potentially affect health outcomes for millions. Yet the quality of health information on the Internet is extremely variable and difficult to assess. Thus the choice of appropriate evaluation criteria for the information is both crucial and challenging. This problem is generally recognized. There is no consensus, however, on how to resolve the problem, and there remain no uniform guidelines for quality assessment of Web-based health information for consumers. A number of websites display rating schemes, and there are "stamps of approvals," but there is little explanation of how those schemes were developed or how the ratings have been applied. The Health Summit Working Group addressed this problem by developing a set of criteria for use in assessing the quality of health information on the Internet. Purpose The purpose of this paper is to provide a brief overview and explanation of the criteria developed by Health Summit Working Group. The ultimate aim of the group’s efforts is to improve the quality of health information on the Internet, and the group regards the development of these criteria--designed to serve as the basis for decision making and tool development--as an essential step toward that goal. The intended audience for this document includes the general public, policymakers, health professionals, and providers of Web-based health information. It is hoped that organizations and individuals reviewing these criteria will use them to assess the quality of health information on the Internet. Scope The criteria presented herein were developed by focusing on health information websites intended primarily for the general public, and do not address information on mailing lists, listservs, bulletin boards, chat rooms, etc., where it is easier to post information with greater anonymity. The emphasis is on the quality of the information provided. Methods and Process The Health Information Technology Institute convened a one-day Health Summit Meeting at Mitretek Systems in McLean, Virginia, on Friday, November 22, 1996. To ensure objectivity in the development of the criteria, 18 individuals with diverse backgrounds, including leaders of medical schools, libraries, pharmaceutical associations, government agencies, professional medical associations, and consumer groups, were invited to attend. The meeting consisted of a series of presentations, open discussions, and breakout groups, and resulted in the initial identification of potential criteria and their strengths and weaknesses, as well as a list of items for future action. Additionally, participants in the meeting developed the preliminary draft of this policy paper that was presented for comment as described above (http://hitiweb.mitretek.org/docs/criteria.htm) The next step in the process was to rank these initial criteria in order of importance. Members of the Health Summit Working Group and other individuals and organizations were invited to provide their suggested rankings. The criteria were categorized as either "Essential," "Important," or "Desirable," and these ratings served as a starting point for further refinement of the criteria at the second Health Summit Meeting. The second Health Summit Meeting was convened on Friday, October 17, 1997, at Mitretek Systems. Participants included those who attended the first meeting, as well as many newly recruited individuals. The purpose of this meeting was to develop an implementable set of criteria (based on the policy paper) for use by the general public. The participants first reviewed and identified those criteria capable of being implemented, and then further refined these criteria into a usable question-answer rating format to be incorporated into a tool developed by the Health Information Technology Institute. In breakout sessions, participants worked on an implementation strategy and identified next steps, which included testing of the tool by volunteers from each of four working groups comprising consumers, librarians, health-care providers, and website developers, respectively. The results of the preliminary tool testing and refinement were then presented at the third Health Summit Meeting, held May 18, 1998, at Mitretek Systems. The focus of this meeting was on the refinement, analysis, and testing of the tool, as well as initial steps toward the creation of a "gold standard" for high-quality health information websites. Evaluation Criteria The Health Summit Working Group selected, defined, ranked, and evaluated seven major criteria for assessing the quality of Internet health information: credibility, content, disclosure, links, design, interactivity, and caveats (advisories). Criteria for Evaluating Internet Health Information
Credibility To determine the credibility of Internet health information, one must consider its source, currency, relevance/utility, and editorial review process. Source. The source of medical information is the premier criterion for its credibility and quality. Therefore, a site should display the name and logo of the institution or organization responsible for the information, as well as the name and the title of the author(s), if relevant. Additionally, the qualifications/credentials of the organization and author(s), along with any relevant personal or financial associations or other real or potential sources of bias, should be disclosed. Disclosing sponsorship and the nature of such support can help consumers assess the motivations of the provider of the information (e.g., if the site is advertising a product or service) and any potential conflicts of interest. Currency. The date of the original document on which the information is based and the date of posting on the Web should be provided so the user can judge the timeliness of the information. Although the date of posting does not necessarily show that the information provided is correct or up to date, it does serve as an indicator of currency. Relevance. Users need to know the content of a site corresponds to the information it purports to offer. Site Evaluation. In an academic community, the peer review process is used to ensure the validity and quality of the information presented in papers and reports. The general public, however, is more likely to understand a “seal of approval” from an individual or group commonly perceived as credible. Sites should indicate whether the information provided has been subjected to review, and if so, describe the process and the individuals involved. Content The content of health information on the Internet must be accurate and complete; an appropriate disclaimer should also be provided. Accuracy. Given that accuracy of content is based on evidence and its verification, the site should identify the data that underlies the conclusions presented. Clinical or scientific evidence that supports a position should be clearly stated. The framework of the study should also be described in language the lay person can understand. And users need to be aware that testimonials are not evidence. Disclaimer. A disclaimer describing the limitations, purpose, scope, authority, and currency of the information should be provided. To ensure accuracy and avoid plagiarism and copyright violation, sources of the information should also be disclosed. The disclaimer should emphasize as well that the content is general health information, not medical advice. Completeness. The treatment of a topic should be comprehensive and balanced. Users should be wary of one-sided views with critical information missing. Pertinent facts, negative results, and a statement of any information not known about the subject should be included. Disclosure Websites should provide appropriate disclosures, including the purpose of the site, as well as any profiling/collecting of information associated with using the site, so users can understand the intent of the organization or individual in providing the information. Purpose. The mission or purpose of the site should be clearly stated, and the information provided should be appropriate to that mission or purpose. Profiling/Collection of Information. Websites request and use information for purposes of which the user may be unaware. It is critical that users be informed of the collection, use, and dissemination of any information they may be providing in visiting the site. Only then can they make an informed decision to provide the information and/or approve of its eventual use. Links Especially critical to the quality of an Internet site are its external links--connections to other internal pages or to external sites that form the web-like structure of information searches within and among sites. There are four criteria for evaluating the quality of links: selection, architecture, content, and back linkages. Selection. The selection of links is made at the originating site. The person or group responsible for link selection should have the expertise and credentials to evaluate critically the appropriateness of those links. It is also important that the original and linked sites target a set of readers with similar characteristics. Architecture. The architecture or design of pointers to linked sites is important for ease of navigation: whether there are timely escape mechanisms during side searches, whether the user can easily find his or her way backwards and forwards, and whether the structure is apparent and logical to the reader. Image-based icons and textual identifiers should be meaningful and consistent. Content. The content of the links should be accurate, current, credible, and relevant. The content of the originating site is enhanced if it includes links to high-quality sites; on the other hand, links to poor-quality sites indicate a lower-quality originating site. Users should be alerted when they are about to view an external site. Means that can be used for this purpose include information relating to the linked source, provided before the user clicks to the site, and use of transition screens. Back Linkages. Back linkages are links to one website from another. Many websites track and publish back linkages for the purpose of enhancing their credibility and marketability. The best way to evaluate back linkages is to examine the context in which they are used, that is, their purpose, relevance, credibility, and authority, as well as any associated bias. Design The design or layout of the website, including graphics and text, as well as links, is important to the effective delivery and use of any Web-based information, even though it does not affect the quality of the information per se. The design of websites can be evaluated in terms of accessibility, logical organization (navigability), and internal search capability. Access. Websites should be accessible by the lowest-level available browser technology. Other features to improve access include options for accessing the information when multimedia browsers are not available, as well as options for enabling use by the hearing and seeing impaired. Logical Organization (Navigability). The best websites are clearly focused on their purpose and target audience. They are simple, internally consistent, and easy to use. Cross-references are provided to aid the user in comprehending the overall structure of the information. The composition of the information reflects an awareness of reading level, language, and the need for a balance of text and graphics, color and sound. Internal Search Capability. An internal search engine is a highly desirable component for most websites with depth and breadth of content. The scope and function of the search engine--what it covers and how it works--should be clearly described. The search engine should be capable of searching specified content by keyword or search string and retrieving only relevant materials. It should also have a user interface that is easy to understand and use. Interactivity Websites should include a feedback mechanism for users to offer their comments, corrections, and criticisms, and raise questions about the information provided. This makes the website accountable to its users. If a site provides a chat room, allowing information to be exchanged among many individuals, an indication of whether a moderator is present should be provided, together with a warning that the information may not be accurate. If a moderator is present, his/her expertise and affiliations, as well as the source of his/her compensation should be identified. When a website provides an interactive service, such as tailoring information to the user based on clinical algorithms, the algorithm used should be posted, including its developer and the site’s affiliation with the developer. Caveats Sites that market services and products have different agenda then those that are primary content providers. Summary The Internet presents a powerful mechanism for helping users improve their health-care decision making by providing easy and rapid access, exchange, and dissemination for enormous amounts of health information. Yet users must be aware of the potential for misinformation and recognize the critical need to assess the quality of the information provided. Content providers must be encouraged to develop and post high-quality information, and policymakers and health-care professionals must be educated on this important health issue. The Health Summit Working Group has developed this set of criteria to address this critical need. These criteria are intended as a resource for users seeking health-related information on the Internet, and should aid in evaluating information to determine whether it is usable and credible. Action Plan Individuals and organizations reviewing this policy paper are encouraged to become involved in this important effort at some level, whether it be supporting and/or endorsing this policy paper, adopting the use of the criteria, joining the Health Summit Working Group, educating others on this topic, or participating in the tool development and testing process. Contact Information For more information on the Health Summit Working Group or to join, visit our homepage - http://hitiweb.mitretek.org/hswg Post comments regarding the policy paper directly to http://hitiweb.mitretek.org/docs/criteria.html Or contact Dr.Helga E. Rippen directly: HITI, 7525 Colshire Drive, McLean, Virginia, 22102-7400, Phone (703) 610-2442; Fax [(703) 610-2022 ]; or E-mail [hrippen@mitretek.org]. Acknowledgements Chair: Helga Rippen, MD, PhD, MPH, Health Information Technology Institute, Mitretek Systems Lead - Librarian Working Group: Roger Guard, University of Cincinnati and Association of Academic Health Sciences Libraries Lead - Consumer Working Group: Marshall Kragen, JD; Next lead to be announced Lead - Health Care Provider Working Group:; Patricia Byrns, BSN, MD, University of Colorado Health Sciences Center;Bill Silberg, Journal of the American Medical Associaiton Lead - Website Developer Working Group: Denise Silber, MBA Lead - Products and Services Working Group: Richard Cleland, Federal Trade Commission Patricia A. Abbott, MS, RN-C, University of
Maryland School of Nursing; American Nursing Association | Top of Page | Position Statements | Policy Forum | IEEE-USA | Last Updated: 23 November 1999 Permission to copy granted for non-commercial uses with appropriate attribution. No rights claimed with respect to the Health Summit Working Group Policy Paper reproduced as an appendix to this position paper. |
![[IEEE-USA Position Statement]](/images/index/ieee_position.gif)