Michael E. Mispagel, Ph.D.
Office of Vice President for Research
August 24, 1998

The University of Georgia is committed to its mission of teaching, research and service. This institution is known throughout the world as a strong research institution. To maintain that reputation, it is imperative that UGA produce the highest quality research possible, research that is above reproach and which will stand on its own in any forum. The administration of The University of Georgia is committed to that goal and will accept nothing less than high quality research that is reproducible, well documented, and has the highest integrity. Toward that end, this guide is submitted to better train our students, faculty and staff in the principles and practical applications necessary to produce quality research.

The diversity of our research is, of course, great. Many of the suggestions offered here may not be appropriate for research conducted in some disciplines, e.g. Fine Arts. Nonetheless, many of the principles expressed in this document will cross over disciplines and be applicable regardless of the research endeavor conducted.

This guide will not only provide a standard of quality control for research at UGA, but through the definition and pronunciation of high quality research expectations, it is anticipated that adherence to the principles espoused in this guide will foster an environment for that research to flourish under common principles of quality.

Demonstration of high quality research practices makes the University community more competitive in the global marketplace both for faculty and staff competing for research dollars, as well as for students seeking employment after their academic and research training at The University of Georgia.

For further information on Quality Assurance, contact The University of Georgia's Quality Assurance Manager, Michael E. Mispagel, Ph.D. at 706-542-5729 or by email at mispagel@vet.uga.edu.

Chapter 1

• Quality Overview
• Ethical Standards in Academia
• The "Cost" of Quality
• Standards Organizations
• University Resources and Policies
• Technology Transfer and Patent Issues
• Animal Care
• Human Subjects
• Biosafety
• Quality Assurance

Chapter 2

• Introduction to Quality objectives
• Basic QC/QA requirements: Level 1
  • Grant Proposal Process: Preparation and Assistance
  • Study Documentation Types, Use of Forms, Paper Trail Philosophy
  • Personnel Qualifications/training
  • Reagent, Solution, and Hazardous Chemical Labeling and Storage
  • Equipment Documentation, Calibration, Metrology
  • Sample Labeling and Tracking
  • Handling and Tracking of Testing, Reference and Control Materials
  • Standard Operating Procedures and Their Use
  • Data Handling
  • Use of Checks, Standards, Blanks, Spiked Samples
  • Raw Data Ownership, Storage and Availability
• Basic QA Checklist

Chapter 3

• Basic QC/QA Requirements: Level 2
  • Introduction to the GLP Mind-set
  • Study Management and Facility Management
  • Protocol Development and Amending
  • Study Documentation Requirements and Error Corrections
  • Computer and Software Validation and Use
  • Quality Assurance Unit
  • Data Storage and Retrieval, Archiving, Raw Data Availability
• QA Checklist - GLP

Appendices

• Facilities Serving the UGA Research Community
• Draft Protocol Checklist
• Facility Inspection Report
• Analytical Chemistry Audit Form

References

*Or download the Microsoft Word version here.*

Chapter 1

Quality Overview

There are many benefits of good quality control and quality assurance to the researcher. Research by its nature is a very systematic process. Inherent in that process is the care and organization needed to obtain the desired results or to test a given hypothesis. Good documentation of the steps taken, the procedures used, the equipment relied upon, the observations taken will not only aid the researcher in the data analysis, but will provide a high level of confidence that the data were free of contamination by unknown variables, that the interpretation of those data is accurate, precise, and complete, and that the effects of unanticipated occurrences are minimized.

Without question, good documentation of a study benefits the researcher during publication preparation. Complex studies require contributions by many individuals. Without good organization and quality control built into the system, the chance of unintentional errors occurring increase, and the opportunity to misinterpret procedures, results or observations increase. A systematic quality control program will minimize those possibilities and will result in more defensible conclusions.

The competition for funding opportunities never seems to diminish. Demonstration of high quality research practices will provide the researcher with a competitive advantage over those who are unable to cite the ability to comply with given standards, whether they be self-imposed standards such as this guide, or federal mandates such the Good Laboratory Practice regulations.

It is the duty of this institution to teach and train students for entry into the marketplace. Oftentimes the standards imposed upon those of us in academia do not equate well with those in industry. It is incumbent upon us to prepare students to compete effectively in that arena. One way to do that is to give our students an idea of the standards of quality expected by industry and government in this country and globally. Learning and practicing those standards here in the academic environment will go far in enabling our students, post-docs, and technicians to enter the non-academic work force well prepared to meet and exceed the challenges ahead.

Oftentimes, research conducted at the University results in a product or process which has patentable and marketing opportunities. Both federal and international laws require specific and detailed documentation of the development of the research conducted to produce the product or process. The organization of a research program resulting from implementation of a good quality control program will aid in meeting the requirements to attain legal property rights through patent protection.

Ethical Standards in Academia

The academic community has always prided itself on its high ethical standards in seeking truth through academic endeavors. In practice our faculty train our students in ethical standards through formal classes as well as by demonstrating ethics in their research and publication practices. By maintaining a quality control program emphasizing good documentation, explicit expectations and the acceptance of responsibility for our research efforts, we are able to emphasize the need and the practicality of high ethical standards for ourselves and our research teams. These lessons in self-discipline and ethical standards will carry over in all of life's endeavors and, thus, are appropriate to the academic environment.

The "Cost" of Quality

What is the "cost" of quality? Some would argue that "quality" is inherent in the scientific process and is therefore inseparable from it. Others would argue that extensive documentation suggested by a quality program is only so much unnecessary, time-wasting paperwork. Both arguments probably have valid points, but the question is better asked in reverse. What is the cost of a lack of quality? Data which may be unreliable because of the lack of training of some team members, or because of uncalibrated, questionable equipment, or variable data-collection techniques among staff may present the researcher with a data acceptance, data interpretation, or even a publishing dilemma. Obviously, the discovery of questionable data at any point in a research program can be costly. It is always better, and cheaper, to design quality into a program or research endeavor at the beginning to avoid the high cost of needless repetition.

Standards Organizations

As trainers and educators of tomorrow's workforce, it behooves the University community to appreciate the standards imposed upon industry, both nationally and internationally, and to train our students such that they are appropriately prepared to enter that job market. Quality standards are imposed upon most industries and, in today's marketplace, are critical to remain competitive.

The American National Standards Institute (ANSI), the American Society for Testing and Materials (ASTM), the International Standards Organization (ISO), and the American Society of Quality Control (ASQC) are a few of the agencies which develop standards. Such standards are used by others to determine the qualifications of a laboratory for certification or accreditation. Analytical laboratories conducting environmental assays may soon be subject to certification through the EPA's National Environmental Laboratory Accreditation Program (NELAP). Many international corporations which produce a product or a service are now striving to become AISO 9000" certified to remain competitive in the global marketplace. The common variable among all the quality standards which exist is the need for good documentation and an attempt to meet or exceed the expectations of the customer. The University environment is the place to learn those skills through involvement in high quality research programs.

University Resources and Policies

The Research Policies & Procedures Handbook available from the Office of the Vice President for Research (Phone (706) 542-5969) describes in detail the resources from that office available to the researcher and delineates University policies relevant to the University's research community.

The University of Georgia is committed to maintaining a safe work environment for its students, faculty and staff. The University provides numerous resources to attain that goal. It is incumbent upon the researcher to familiarize him/herself with the laws applicable to their research and research team. Appendix 1 lists offices which are prepared to assist you with your research needs. A more detailed description of many of these offices is in the above cited handbook. Often it is necessary to obtain approval from one or more of the compliance offices before grant proposals can be submitted or before funds can be released to the researcher. Early contact with the compliance office staff to determine a project's needs is suggested to avoid delay of project funding.

Technology Transfer and Patent Issues

The University of Georgia's Research Foundation, Inc. (UGARF) assists researchers with intellectual property and technology transfer. Chapter II of the "Research Policies and Procedures Handbook" discusses in detail the policies of the institution. UGARF should be contacted (Phone (706) 542-5944) for additional detail about copyrights, patent protection, licensing, and marketing an invention.

Clear and unambiguous documentation is critical in obtaining a patent or marketing an invention. Most inventors do not set out to simply invent something from scratch. Inventions are usually "discovered" during the course of other work. When that occurs, it is important to have appropriate documentation practices in place to substantiate the claims of the researcher. This section provides some information regarding the documentation necessary for a successful patent application. Patent law requires that an invention be new or novel, useful, and non-obvious.

The laboratory notebook is of extreme importance to the individual wishing to obtain patent protection for an invention. It provides 1) the information needed to decide whether or not to apply for a patent; 2) the information needed to complete the application for the patent; and 3) evidence of the invention's conception and when it was first demonstrated. The invention's "conception" is when the invention was first formulated in the mind of the inventor. "Reduction to Practice" is the term used for the first experimental demonstration providing the evidence that the invention works.

The notebook must answer the following questions: 1) What is the invention? 2) When was it made and where is the experiment recorded? 3) Has the invention been disclosed publicly? 4) What is known about prior work in the field? and 5) How complete are the data?

An "Invention Disclosure" includes the following: Names and addresses of inventors, date of the conception of the invention, the date when a working model was demonstrated, a description of the invention, an explanation of why the invention is novel, the uses of the invention, a list of how and when the invention has been discussed or shown to others. The notebook will contain the original notes and data needed to substantiate the claims of the inventor by showing 1) the date of the conception of the invention, 2) the continuity of the record, 3) appropriate witnessing, 4) disclosures of the invention to the public, and 5) speculation about the usefulness of the invention.

If the patent is challenged by another researcher and becomes the subject of an interference proceeding, you will need records to substantiate the date of conception and evidence of your "diligence in reduction to practice". That is, you must show that you worked continuously on your invention and did not abandon it. If a lengthy period occurs when no direct data entries will be made, you can show "diligence in reduction to practice" by recording in your notebook, for example, when you thought about the project, that you were waiting for samples, that you were ill or on vacation, etc.

Have your laboratory notebook witnessed frequently. A witness must be "a person skilled in the art to which it pertains..." and must be able "to make and use the same." Witnesses must be trustworthy, be willing to testify on your behalf and be available for a few years. A witness cannot be a co-inventor. The witness must be clear as to what is being witnessed, i.e. "Page read and understood", "Operation of apparatus observed", "Material tested in my presence", "Samples synthesized in my presence", etc. Do not simply write "Witnessed by ___". Have witnesses record in their own notebooks that they witnessed your work. A Patent Office publication emphasizes this point: "Your priority right against anyone else who makes the same invention independently cannot be sustained except by testimony of someone else who corroborates your own testimony as to all important facts, such as conception of the invention, diligence, and the success of any tests you may have made."

If you disclose your invention to anyone else, note that fact in your notebook and under what circumstances you disclosed your invention. Note if someone else contributed an idea to your invention including possible uses for the invention. If you submit a written description to anyone such as a journal, note that fact in your notebook and keep the correspondence. If your invention is orally presented in a seminar or at a conference, note that too. If the invention is discussed at a staff meeting, take your notebook to the meeting in order to make notes directly.

Speculation is alright when you are suggesting novel uses for a new invention. This is important because an invention must be useful in order to be patentable, and because a new use for an established material is patentable. Never suggest that it is not suitable for certain uses.

Specific recommendations from Roman Saliwanchik (1988, pages 20-22) are shown below:

1. Use a bound laboratory notebook with numbered pages to record all experiments.
2. Write the date that the work started in the upper right corner of the page.
3. Make all notebook entries in ink of a single color to facilitate proving the content of the entry.
4. Fully describe the planned details of the experiment indicating the purpose of the experiment and the expected results.
5. If some of the experimental work is done by another person, for example an assayist, the data obtained from the assayist should be recorded in the bound laboratory notebook as soon as they are received by the researcher who submitted the material that was assayed.
6. Write the date that the assay results were received on the same page where the results are recorded, and identify the assay date by reference to a prior notebook page that contains a description of the material upon which an assay was requested.
7. Make entries directly into the notebook as the experiment is carried out and immediately when results are obtained.
8. If an incorrect entry is made, draw only a single line through the incorrect entry; there should be no attempt to obliterate or erase the incorrect entry.
9. Enter all conclusions and observations about the experiment into the notebook; even negative experimental results are important and, therefore, should be included in writing in the notebook.
10. Immediately after each page of the notebook is completed, it should be initialed and dated by the person actually doing the experiment; this person may very well be a laboratory assistant.
11. Each page of the notebook should also be initialed and dated by the person who supervised the person actually conducting the experiment.
12. The recording of the conclusion of an experiment should be followed immediately by the signature of the experimenter; and a witness, i.e., a person who observed and understood the experiment or a person to whom the experimenter disclosed the experimental details and who understood the experiment, should sign thereafter.
13. If a notebook page is not completely filled out, a diagonal line should be drawn through the blank portion.

Animal Care

The University is committed to the humane use of research animals and complies with all state and federal laws pertaining to animal welfare and protection including the Animal Welfare Act and the Guide for the Care and Use of Laboratory Animals (NIH 1996). Portions of UGA's animal facilities are accredited by the American Association for Accreditation of Laboratory Animal Care (AAALAC). Researchers who use animals in their programs must explain and justify their use of animals in an Animal Use Proposal (AUP) which must be reviewed and approved by an Attending Veterinarian and by the Institutional Animal Care and Use Committee (IACUC) prior to the use of animals in a research program. The AUP is valid for three years but must be reviewed annually. Any modifications to the AUP must be approved by the IACUC. It is the goal of the IACUC to assure that any pain or suffering to research animals required for research is reduced or eliminated, and to assure that the numbers of animals used in research projects are necessary.

The IACUC inspects the University's animal facilities twice a year to assure that the facilities and the animal care meets the expected standards. In addition, a USDA Veterinarian similarly inspects the animal care program and the facilities at least once a year.

Investigators and technicians who will be responsible for the care and use of research animals must have appropriate documented training prior to their use of animals. Contact the Animal Care and Use Office at (706) 542-5933 for additional information about obtaining this training.

Human Subjects

Similarly, the Human Subjects Office (Phone (706) 542-3199) tracks and approves all student and faculty projects using humans as research subjects. Federal law requires that appropriate informed consent be obtained prior to initiating any studies using humans and that an Institutional Review Board review and approve those projects prior to their start.

Biosafety

The Biosafety Office (Phone (706) 542-0112) will assist those researchers who will work with pathological organisms or recombinant DNA/RNA. A Memorandum of Understanding (MOU) will be completed after the facility and project have been inspected and have met all federal and state safety requirements regarding the organisms to be handled.

Quality Assurance

The Quality Assurance office (Phone (706) 542-5875) is in place to assist University researchers produce the best quality data possible. In particular, this office will train staff in the principles and application of the federal Good Laboratory Practice regulations when those are required by a sponsor. The QA office will also assist in the preparation of a study protocol, writing of SOPs, facility inspections, as well as data and final report audits. The QA office maintains a web page related to quality assurance issues and documents at http://www.ovpr.uga.edu/qau/.

Return to the Quality Assurance Homepage

Chapter 2
Introduction to Quality objectives

This institution supports research of many kinds, from many disciplines, and of extremely variable sophistication. Keeping that in mind, it is difficult to generalize too broadly about specific criteria common to all. However, there are some guidelines which are applicable to, at least, the disciplines which collect data of some kind for subsequent analysis and eventual report. It is those types of research efforts which will be most aided by these "research quality control guidelines".

For convenience, we will divide the discussion of research expectations between two types of research. The first is by far the most common type of research conducted at the University. All research which is NOT conducted under the mandate of the federally mandated Good Laboratory Practice regulations would fall into this category. This would include research conducted by students, postdoctoral associates, technicians, and faculty under a "peer-review" modus operandi. Most would agree that research conducted by students, though under the guidance and training of a faculty member whether teacher or mentor, should be conducted at the same level of quality as senior faculty whom they should emulate.

The second type of research to be discussed in the next chapter is that which falls under the mandate of the federal government's Good Laboratory Practice regulations which is required for data being submitted to a federal agency, usually the Food and Drug Administration [21 CFR Part 58] or the Environmental Protection Agency [40 CFR Part 160], that support or are intended to support applications for research or marketing permits for products regulated by those agencies. However, many other federal agencies now require compliance with these regulations. The requirements for documentation are much more strict than you would expect for basic research data submitted for peer review.

Return to the Quality Assurance Homepage

Basic QC/QA requirements: Level 1

Grant Proposal Process: Preparation and Assistance

All research begins with a hypothesis or subject of interest usually outlined in a grant proposal, study plan, or contract with a funding agency or sponsor. Assistance with sources of funding and grant preparation can be obtained from the Office for Sponsored Programs (Phone (706) 542-5939). Many facilities for grant preparation are available on line at http://www.ovpr.uga.edu/sponprog/. Most applications for grants are specific as to the information and format required.

At a minimum, the study plan should describe the proposed research including a description of the organisms, subjects or materials to be worked with, a justification for the study, the methods to be employed, the observations to be made, the records to be kept, the statistics to be used, and the responsibilities and obligations of the personnel who will be involved in the study. The lack of a clear understanding of the study and the duties of all individuals involved will inevitably lead to misunderstandings which may compromise the integrity of the study.

Study Documentation Types, Use of Forms, Paper Trail Philosophy

The basis of all quality work is good documentation. Every study should have an accepted method of documenting the activities, data, and findings of the investigation. This documentation can be in the form of a laboratory/study notebook or a binder of pre-printed forms specific to the study. Whatever form they take, steps must be taken to assure that data is not lost or misplaced and to be able to recognize that fact if it occurs, i.e. a missing page number. Use of bound notebooks with pre-printed page numbers works for many cases, as does paginating individual loose-leaf forms which contain the study name or number and which are kept in a binder. Clear identification of the owner of the material should be conspicuous.

Given the fact that all scientific accomplishments should be reproducible, researchers must develop the philosophy and practice of maintaining a paper trail to permit the reconstruction of the study absent the present investigator(s). Thus, the notes taken during an investigation should be complete, clear, and unambiguous. Full equations with units should be written in the notebook, rather on a piece of scrap paper. Each step of a procedure should be recorded unless you are following a written standard operating procedure which is referenced. Re-read the section on the hints for maintaining a quality laboratory notebook for patent protection.

Errors in data collection and recording are common occurrences. Mistakes should be identified by a single line through the error, and the researcher's initials and date of change placed next to the correction to identify the one making the correction and when it was done. You should never obliterate an error, erase it, or use correction fluid to correct a mistake since the "error" may need to be referenced at a later time. It is always preferable to record all data in indelible ink, rather than pencil.

Because it is not uncommon for data to be lost or destroyed inadvertantly, copies of raw data should be made periodically and stored in another location.

Personnel Qualifications/training

All employees of the University should have a personnel file which documents the qualifications and training attained for the job. Employees should be comfortable in their knowledge of how to conduct a procedure, use a piece of sophisticated equipment, or interact with a research subject. Toward that end, it is useful for the employee to keep a list of procedures (SOPs) to be performed, and equipment which is expected to be used. Once "trained" in the procedures or the use of the equipment through the use of the SOPs, the employee should be watched performing the operation or procedure by the employee's supervisor. Once both individuals are satisfied that the procedure has been learned to the satisfaction of both, they can initial and date the document to certify that proficiency has been attained. This simple document then serves as documentation of training which will assure that both the employee and the supervisor are comfortable with the skills mastered before work begins. The potential embarrassment of a costly mistake due to a lack of adequate training before data collection is thus avoided.

Implementation of training documentation should be made easier since the Georgia Right-to-Know law already mandates that those state employees handling or working with hazardous chemicals have documented training on the handling, properties, and hazards associated with the chemicals to which they may be exposed. Each campus unit has a Right-to-Know Coordinator who oversees this program. Ask your department's safety representative for detailed information regarding this program and how it can help you develop your laboratory's training documentation.

Reagent, Solution, and Hazardous Chemical Labeling and Storage

Label all reagent, solution, and hazardous chemical secondary containers with at least the chemical name, concentration (if appropriate), date prepared/opened, expiration date, and initials. State law requires all hazardous chemical containers to have a hazardous chemical warning label affixed unless it is intended only for short-term storage (one week or less). This warning label must be either an NFPA or HMIS hazard warning label indicating the health, flammability, reactivity, or other hazard rating associated with the chemical. Such labels are available from Central Research Stores.

All hazardous chemicals must be stored in well ventilated areas with other chemicals which are compatible. Do not store chemicals alphabetically except within a compatible class. Use appropriate storage cabinets, such as flammable, acid, or corrosives, when appropriate. Do not use non-explosion proof refrigerators or freezers to store volatile chemicals such as solvents or ether.

Equipment Documentation, Calibration, Metrology

Just as you wouldn't knowingly publish false data, neither should you accept data derived from an instrument which has not been properly calibrated and maintained. Blindly accepting data from an instrument which you know little about can be foolhardy. Each piece of equipment should have an associated log book for documenting any changes, maintenance, or calibration conducted, including the date and the name of the person doing the work. Without a written record such as this, there is no way of knowing if the instrument will provide accurate data or if anything was done to the instrument to alter its output. Electronic balances should have their calibration checked daily, when in use, with standard calibration weights to verify the instrument's reliability. A written record of calibration checks assures the researchers using the equipment that the data the equipment provides is accurate at that time. There is no point in recording data from equipment which may provide questionable data because of poor maintenance or a lack of proper calibration. The researcher should always look for documentation which answers the question, "How do you know it works?" before relying upon the data produced by a piece of equipment.

Sample Labeling and Tracking

Lost or missing samples can be more than simply embarrassing. Their loss could affect the outcome of a study. Care must be taken to label all samples collected with the study name/number, the date, the collector's name or initials, and a unique sample number. Prepare the sample labels and attach them to the sample containers prior to sample collection to save time and confusion.

Handling and Tracking of Testing, Reference and Control Materials

Your test material, reference standards and control materials should be handled with care and with a consideration of the possibility of contamination. Good separation of these materials from each other and from the collected samples will help to minimize this possibility. When collecting samples which have been exposed to a range of concentrations, collect the control samples first, followed by the lowest concentration and proceeding to the highest concentration. Clean collecting equipment between sample collections from different concentrations to avoid cross-contamination.

Standard Operating Procedures and Their Use

The use of written Standard Operating Procedures (SOPs) is strongly encouraged to minimize the systematic errors associated with the collecting of data by a number of people each of whom may do things in a slightly different manner. SOPs should be written as a series of steps for the successful conduct of a standard procedure. By following the SOP, steps are not inadvertently left out, altered, or conducted in the wrong order. Familiarity with the SOP will allow all the staff to do the procedure in the same way thus minimizing the chance for error from this source. As new techniques are developed or procedures are modified, the SOP should be revised and updated while retaining the outdated SOP in an historical archive so that you know how a procedure was done at a previous time.

A typical SOP will have a title, SOP number, a stated purpose, and the steps of the procedure in outline form. The lab supervisor or a senior researcher and the author should sign-off on the SOP and indicate the date when it is to be effective. SOPs should be written by the people actually doing the procedures, rather than the senior staff, because those individuals know best how a procedure is conducted. It should be written in a practical fashion, without ambiguity or complexity so that it can be used as a training document for new staff learning the procedure.

Data Handling

Only trained and authorized people should be collecting data or recording notes in a study notebook. The notebook should have a signature page in front where everybody who will use that notebook can place their signature, printed name, and initials so that initials on subsequent pages can be associated with a complete, legible name.

To reduce the possibility of error in the notebook, do not transcribe your data from scrap paper to the lab notebook. Transcription errors are very common but difficult to find. Always record your data directly into the lab notebook. The first recording of the data is the Araw data@ which will be used to reconstruct the study for data analysis and publication purposes. The study notebook is not expected to be without some grime and wear, especially if the study is a field study or a study of long duration. It is much more important to have a study free of transcription errors than to have a notebook free of smudges, coffee stains, and corrected errors. Never throw away any "raw data".

Use of Checks, Standards, Blanks, Spiked Samples

Most research investigations require the use of a "control" group or "untreated check" with which to compare the group receiving the treatment. An analytical chemist will use known standards, blank samples, and spiked or "fortified" samples to compare with his unknown. The size of this control group will vary from field to field, but generally one must consider the type of control samples as well as the size or number of control samples to use as representative of the control population. This is a statistical quality control problem unique to each research project. Review the number, frequency, and type of control samples, standards, field blanks, etc. before beginning a research project. Consultation with a statistician in your field prior to the start of the study may be appropriate.

Raw Data Ownership, Storage and Availability

Ownership of data collected as an employee of The University of Georgia is subject to policies of The University as presented in Section IV-2 of the Research Policies and Procedures Handbook. Data rights applicable to grants generally are dictated in a contract or granting guidelines. UGA or UGARF hold copyright privileges to the presentation of scientific data produced in the performance of duties as an employee of UGA.

Original data collected by a student for a student class project belongs to the student. But data collected by a student being funded by a senior investigator belong to that senior investigator. Students who are leaving the University after conducting research should arrange to take copies of their laboratory notebooks and/or data files while leaving the originals in the possession of their major advisor unless other arrangements are made. Generally, original copies of data should not be removed from the University.

Keeping a second copy of research records in a secure place such as with a major advisor/supervisor or even at home is recommended. Lost or misplaced records or notebooks are not uncommon.

Original data should be stored in a dry, cool, secure area. The data should be sufficiently labeled and identified by project name/number, investigator(s) name, dates, etc. that access for subsequent reference is possible.

There is no specific time limit for which general research data must be archived. But, generally, it is prudent to maintain data for at least five years after the material has been summarized and published. Many investigators have a very difficult time ever disposing of research data.

Basic QA Checklist

Study Name:              

Date:

(Initial and date as appropriate)

Return to the Quality Assurance Homepage

Chapter 3

Basic QC/QA Requirements: Level 2

There are other types of research conducted at The University of Georgia which, by law, requires more stringent documentation. These are studies which will support an application for a license or permit from a federal regulatory agency such as the Food and Drug Administration (FDA) or the Environmental Protection Agency (EPA). Historically, these studies investigated the safety of a proposed product such as a drug, medical device, pesticide, or food additive. Virtually all non-clinical data submitted to these agencies must comply with the Good Laboratory Practice Regulations as promulgated in the Code of Federal Regulations (FDA: 21 CFR Part 58, September 4, 1987; EPA: 40 CFR Part 160, August 17, 1989). You can access these documents at the UGA Quality Assurance home page at http://www.ovpr.uga.edu/qau/. These standards are recognized throughout the world in Memoranda of Understanding between the U.S. and other major developed countries as representing the minimum documentation required to assure the integrity of the data to the regulatory agencies as well as to the consumer.

Because of the widespread use and acceptance of the Good Laboratory Practices (GLPs) in industry, many other federal funding agencies also require adherence to these standards including the U.S. Forest Service, USDA, Soil Conservation Service, and others. In addition, clinical trials of drugs must comply with a similar set of regulations called the Good Clinical Practices, and, once a product is developed and approved for manufacturing, that process must comply with the current Good Manufacturing Practice regulations. All of these regulations are mandated and enforced so as to assure the public of the safety, consistency, and efficacy of the products they are consuming. In addition, they are in place to minimize the potential for systematic error in the generation and collection of data and to reduce the possibility of scientific fraud.

Our brief discussion of these principles here is not only for those who will actually conduct such studies at this institution and thus, must comply with the letter of the law. Those individual research teams will receive detailed training in the GLPs and close scrutiny of their facilities, personnel, and studies throughout the research. This chapter is intended more for those who do not need to comply, but who would like a higher goal or standard of quality research to attain. A great deal of what is in these regulations is also good common sense, and, not coincidentally, good science. And, lastly, our discussion of these principles is to help train those students who will compete in the industrial job market so that they will be prepared to conduct research according to the mandates of quality research imposed upon them.

Introduction to the GLP Mind-set

All of the principles of study conduct and quality control for research studies discussed above pertain to a GLP study as well. But, in some areas, more stringency and organization is required and forced upon the researchers. First, the data has a different purpose and review process for GLP studies. Data generated from GLP studies are generally compiled by a sponsor such as a drug company and are eventually submitted as part of a registration package to a federal agency. Those data will be reviewed by federal auditors and eventually a decision will be made as to the adequacy of the data and whether or not registration should be granted. This process may take many years during which staff personnel from the original studies will probably have turned over many times. Those who review the study material will be attempting to reconstruct the study many years after the work was done using the data records alone. Those individuals are also trained to be skeptical about claims of efficacy and must search for adverse reactions to the test materials to protect the public and the environment. Thus the burden of proof and clarity of documentation is the responsibility of the original investigative team.

This differs significantly from the data generated from typical academic research pursuits which generally are reviewed as summary data in a manuscript through the peer-reviewed publication process. The research may be the same, but the mental discipline required for documenting that research is not. The change in mind-set required for GLP investigations asks the researchers to continuously question themselves and the system in which they are working and to produce the documentation to answer the most common question, "How do I know?". It requires the researchers to assume the worst of their data, their staff, their equipment, etc. unless and until their documentation proves otherwise. Considering that there may be a legal challenge to their data someday, that philosophy seems prudent.

Study Management and Facility Management

A research project in academia may have a number of co-Principal Investigators, each receiving some credit for the grant and each assuming variable responsibilities for the project. A GLP project may have similar co-PI's, too, but in addition one of the PIs must be the single designated Study Director. The Study Director is the one individual responsible for the overall conduct of the non-clinical laboratory study and is the single point of study control. The study director (FDA '58.33) shall assure that:

a) The protocol, including any change, is approved and followed;
b) All experimental data, including observations or unanticipated responses to the test system are accurately recorded and verified;
c) Unforeseen circumstances that may affect the quality and integrity of the nonclinical laboratory study are noted when they occur, and corrective action is taken and documented;
d) Test systems are as specified in the protocol;
e) All applicable good laboratory practice regulations are followed;
f) All raw data, documentation, protocols, specimens, and final reports are transferred to the archives during or at the close of the study.

The Study Director will sign a compliance statement in the final report stating exactly where the study practices differed from the GLP regulations (EPA '160.12).

In addition, the testing facility management have specific responsibilities (EPA '160.31) including:

a. Designate a study director as described in '160.33 before the study is initiated.
b. Replace the study director promptly if it becomes necessary to do so during the conduct of a study.
c. Assure that there is a quality assurance unit as described in '160.35.
d. Assure that test, control, and reference substances or mixtures have been appropriately tested for identity, strength, purity, stability, and uniformity, as applicable.
e. Assure that personnel, resources facilities, equipment, materials and methodologies are available as scheduled.
f. Assure that personnel clearly understand the functions they are to perform.
g. Assure that any deviation from these regulations reported by the quality assurance unit are communicated to the study director and corrective actions are taken and documented.

The Quality Assurance Unit will submit inspection findings to management and the study director for their information and response, if needed.

In industry it is easy to identify the testing facility management. In academia, it is often not very clear who should be responsible for these duties; it might be a department head, dean, director, or an associate vice president. Such a determination of who will assume this role should be made early in the project development stage such as during protocol development.

Protocol Development and Amending

Every GLP study must have a written, approved protocol. A protocol differs from a grant proposal in that the protocol does not need to justify the project since it is not intended to be a fund-raising tool. Rather, it shall indicate the objectives of the study, identify the testing facility and indicate the responsible parties, describe the test system, the experimental design, the observations to be made, the records to be maintained, the statistics to be used, etc. A protocol checklist is available in the Appendix. The protocol must be approved by the sponsor and dated and signed by the study director. Any deviation from the protocol must be documented with justification given, and signed and dated by the study director.

Study Documentation Requirements and Error Corrections

Data collected for GLP studies must be recorded promptly and legibly in ink. All data entries should be dated on the day of entry and signed or initialed by the person entering the data.

There will probably be a number of study files which contain numerous documents pertaining to the study. These files may contain telephone memos, correspondence, shipping labels, organizational charts for your unit, internal memos, quality assurance findings. You may, for example, have files of equipment maintenance/calibration records, SOPs, environmental/weather data, training records, animal room logs, veterinary care reports, pathology reports, analytical lab reports, statistical analysis data, purchase orders, protocol amendments, protocol deviations, unusual occurrences, chain-of-custody records, manifests, interview records, etc. All of these documents must be archived at the end of the study so that the sponsor and the regulatory agencies can reconstruct the study to understand exactly what transpired during the course of the investigation and to understand completely the procedures undertaken, the quality control employed, the integrity of the data collected, and the validity of the results and the conclusions of the study. Needless to say, disorganized or illegible data files will leave a negative impression on those reviewing the data. Thus, the GLP mind set can force a research team to be organized and cautious prior to data collection.

Of course, errors in data entry do occur. Changes or corrections to the data shall be made so as not to obscure the original entry, shall indicate the reason for the change, and shall be dated and signed at the time of the change (EPA '160.130(e)). It is common to use circled code numbers to indicate the reason for a change in a data entry such as the following:

1. Misspelling or carelessness.
2. Instrument was misread.
3. Misunderstood or heard incorrectly.
4. Miscount.
5. Recorded in wrong place.
6. Transposed or out of proper sequence.
7. Illegible.
8. Wrong word or phrase for meaning.
9. Previous error(s).
10. Other unlisted reason. Identify reason for change.

Other error code systems use two-letter codes, such as "WE" for "Wrong Entry", or "WD" for "Wrong Date". Such codes may be easier to remember if there are not too many of them.

Computer and Software Validation and Use

For many researchers, computer use is integral to their programs for collecting, tabulating, manipulating, calculating, and reporting their data. To ensure "a high standard of quality for computer-resident data produced in support of EPA programs", that agency published on August 10, 1995 the "Good Automated Laboratory Practices (GALP): Principles and Guidance to Regulations for Ensuring Data Integrity in Automated Laboratory Operations". The concern of the agency is that too much faith might be put into electronic systems such as "laboratory information management systems" (LIMS). However, not all automated laboratory systems are LIMS. Automated laboratory systems that record data but do not allow changes to the data are not LIMS. The ability to effect changes to original observations or measurements prior to the recording of that data is the factor in determining whether the automated laboratory system is a LIMS and therefore subject to the GALPs.
.
The GALP discusses in detail how to document the validation of the automated data collection system through the use of SOPs on maintaining the security of the system, data entry, verification of input data, data changes, data analysis, processing, storage and retrieval, backup and recovery of data, maintenance of data collection system hardware, and electronic reporting.

Software validation must be conducted according to an SOP to describe what the software is expected to do or the functional requirements that the system is designed to fulfill. All algorithms or formulas used must be listed, and a description of acceptance testing criteria must be written.

All GLP provisions regarding personnel training, data entry, raw data, records and archives pertain as well to automated data collection systems. Thus documented evidence must be available to prove that the personnel are qualified to use the system, that the hardware is adequate for the job, that the software does what it is supposed to do, that adequate security is in place to protect the data, and that all the records are available for audit.

The GALP Guidance document discusses in detail six principles which are inherent in both the implementation of the GALP and its data management policies (pp. 2-1 - 2-2).

1. DATA: Laboratory management must provide a method of assuring the integrity of all LIMS data. Communication, transfer, manipulation, and the storage/recall process all offer potential for data corruption. The demonstration of control necessitates the collection of evidence to prove that the system provides reasonable protection against data corruption.

2. FORMULAE: The formulas and decision algorithms employed by the LIMS must be accurate and appropriate. Users cannot assume that the test or decision criteria are correct; those formulas must be inspected and verified.

3. AUDIT: A critical control element is the capability to track LIMS Raw Data entry, modification, and recording to the responsible person. The capability utilizes a password system or equivalent to identify the time, date, and person or persons entering, modifying, or recording data.

4. CHANGE: Consistent and appropriate change controls, capable of tracking the LIMS operations and software, are a vital element in the control process. All software changes should follow carefully planned procedures, be properly documented, and when appropriate include acceptance testing.

5. STANDARD OPERATING PROCEDURES (SOPs): Procedures must be established and documented for all users to follow. Control of even the most carefully designed and implemented LIMS will be thwarted if the user does not follow these procedures. This principle implies the development of clear directions and Standard Operating Procedures (SOPs), the training of all users, and the availability of appropriate user support documentation.

6. DISASTER: The risk of LIMS failure requires that procedures be established and documented to minimize and manage their occurrence. Where appropriate, redundant systems must be installed and periodic system backups must be performed at a frequency consistent with the consequences of the loss of information resulting from a failure. The principle of control must extend to planning for reasonable unusual events and system stresses.

Quality Assurance Unit

The Quality Assurance Unit (QAU) must be separate from and independent of the personnel engaged in the study (EPA '160.35). The role of the QAU is to assure management that the facilities, equipment, personnel, methods, practices, records, and control are compliant with the GLP regulations. The QAU must maintain a master schedule of ongoing studies and protocols of those studies. The QAU inspects each study at intervals adequate to ensure the integrity of the study and keeps records of those inspections, reports the finding of those inspections to the Study Director and management, and reviews the final study report Ato assure that such report accurately describes the methods and standard operating procedures, and that the reported results accurately reflect the raw data of the study@. The Quality Assurance Unit will also prepare and sign a statement in the final report specifying only the dates that inspections were made and the dates that the findings were reported to management and to the Study Director.

The QAU should be viewed as an independent member of the research team whose goal is to aid the research team to produce the most accurate and defensible product possible within the constraints of the GLPs.

Data Storage and Retrieval, Archiving, Raw Data Availability

It may take many years for a company to compile and submit a regulatory package of completed studies in support of an application. Thus, data storage and retrieval are important. AAll raw data, documentation, records, protocols, specimens, and final reports generated as a result of a study shall be retained. Specimens obtained from mutagenicity tests, specimens of soil, water, and plants, and wet specimens of blood, urine, feces, and biological fluids, do not need to be retained after quality assurance verification {' 160.190}.@ Many companies use a commercial archive facility to handle this task. Records must be retained 1) as long as the sponsor holds any research or marketing permit to which the study is pertinent, or 2) at least two years after the completion of the study if it is not submitted to the agency, or 3) at least five years following the date on which the results of the study are submitted to the agency in support of an application for a research or marketing permit.

QA Checklist - GLP
(In addition to basic QA checklist)

Study Director:

Study Title:
(Initial and date when complete)

Return to the Quality Assurance Homepage

Appendix

Appendix 1

Facilities Serving the UGA Research Community