<set xml=true><silence msec="7000" /  > 
  
Evaluating Sources of Knowledge For Evidence-based Actions in Public Health  <set xml=true><silence msec="1500" /  > 
  
Secretarys Advisory Committee on National Health Promotion and Disease Prevention Objectives for 2020 
 
September 2009  <set xml=true><silence msec="1500" /  > 
 
Purpose of this Brief  <set xml=true><silence msec="1500" /  > 
The Healthy People initiative, coordinated by the Office of Disease Prevention and Health Promotion (ODPHP) under the U.S. Department of Health and Human Services (HHS), provides quantitative health promotion and disease prevention goals and objectives to be achieved in ten-year increments.   Planning is underway for the fourth iteration of national objectives-Healthy People 2020.  To aid in this effort, the Secretary’s Advisory Committee on National Health Promotion and Disease Prevention Objectives for 2020 (the Committee) has been convened to provide advice and consultation to the HHS Secretary. 
In a recent report, the Committee noted that Healthy People has sometimes been described as a catalogue of the burden of ill-health, disability, and premature death.  While Healthy People has set targets for reducing burden, it has not offered guidance on potential actions for achieving these targets, or the relative effectiveness of such actions.  Yet such guidance can be complicated by the fact that standards of evidence vary for different types of health interventions. 
The Committee therefore proposed that Healthy People 2020 should seek to tie goals and objectives to focused, evidence-based interventions that can guide effective action and accountability at the federal, state, and local levels.  Seamless linkages are needed from Healthy People 2020 to existing resources that periodically evaluate and interpret evidence.  Examples of such resources include the Guide to Community Preventive Services, the Guide to Clinical Preventive Services, and Cochrane Reviews.   <set xml=true><silence msec="1500" /  >  
 
Origins of the movement toward evidence-based public health practice  <set xml=true><silence msec="1500" /  > 
Evidence-based public health has its roots in clinical epidemiology and evidence-based medicine (EBM).  During the 1970’s and 1980’s, evidence accumulated that expert reviews and recommendations from expert panels frequently failed to include relevant studies, and produced suboptimal conclusions.  It was not clear what aspects of health care practices were associated with better health outcomes.  EBM was developed in response to this experience.  It has sought to explore which combination of specific services and medical conditions lead to improve health outcomes in actual practice, and for whom.  EBM originated in the management of individual patients, where the best available evidence was combined with patient preferences and knowledge of local resources to improve decision-making.   
More recently, EBM has focused primarily on clarifying aspects of medical decision-making that can be made on a scientific basis, while recognizing that judgments about appropriate treatment often depend on individual factors (e.g., values or quality of life).  Preventive services have been in the vanguard of this movement. In 1984, the establishment of the on US Preventive Services Task Force (USPSTF) built on the work of the Canadian Task Force on the Periodic Health Examination, which had been initiated almost a decade earlier. The general medical community began focusing on EBM in the 1990’s, employing scientific methods to assess which diagnostic or therapeutic strategies would produce the best medical outcomes.   
The foundation of the evidence-based approach is the “systematic review,” epitomized for many by the Cochrane Collaboration meta-analyses. Systematic reviews summarize the results of available, carefully designed and executed studies, and provide an assessment of the quality of evidence and the effectiveness (net benefit or balance of benefits and harms) of health interventions.   They are produced through a rigorous methodical process in which reviewers detail how studies were identified and selected; the extent to which the studies were useful for answering review questions; and how the results of the separate studies were combined to yield an overall measure of the benefits and harms for a particular intervention.  Information gleaned from evidence-based reviews is used to improve the quality of care, such as by incorporating them into evidence-based recommendations, quality improvement metrics and incentives, and clinical decision support tools.  <set xml=true><silence msec="1500" /  >  
What is “evidence-based” public health practice?  <set xml=true><silence msec="1500" /  > 
There has been strong recognition in public health of the need to identify the evidence of effectiveness for different policies and programs, translate that evidence into recommendations, and increase the extent to which that evidence is used in public health practice. Because planning to address population-based health problems typically takes place within a context of limited resources, decision-makers should invest in proven, cost-effective solutions. Evidence for the effectiveness of interventions-such as programs, practices, or policies-can be used to provide the rationale for choosing a particular course of action, or to justify the allocation of funding and other resources.  There is demand for evidence at many levels. For example, practitioners use it for program planning; local managers use it to make decisions about which programs to support; and senior managers within government and organizations use it to set priorities and make policy and funding decisions.   
Evidence-based public health is the development, implementation, and evaluation of effective programs and policies in public health through application of principles of scientific reasoning, including systematic uses of data and information systems, and appropriate use of behavioral science theory and program planning models.  Just as EBM seeks to combine individual clinical expertise with the best available scientific evidence,  evidence-based public health draws on principles of good practice, integrating sound professional judgments with a body of appropriate, systematic research.  <set xml=true><silence msec="1500" /  >   
How is “evidence” defined and evaluated within a public health context?  <set xml=true><silence msec="1500" /  > 
Public health evidence can take many forms.  In a larger sense we can divide evidence for social decision-making into 3 categories 7 The first is scientific information that is independent of context. This is typified by assessment of the efficacy of specific technologies; it answers the question of whether an intervention can work at all.  The second type of evidence is social science evidence, which can be equally rigorous, but is context- sensitive.  An example is economic evaluation, where cost-effectiveness can be determined, but may vary based on how an intervention is implemented, the adherence, and other contextual issues.  The last type of evidence is anecdotal information that is truly local.  In this discussion, we focus primarily on the first type of evidence about whether an intervention can work, and the potential health impact. 
Systematic reviews of high quality studies are an important way to assess specific interventions. Yet many interventions, particularly policy interventions, are not amenable to traditional, systematic evidence reviews.  While health impact assessments (HIAs)8 are not the principal focus here, it is important to recognize that they offer another evidence-based approach that uses the best-available information to inform decisions. HIAs use established methodologies and modeling techniques to provide an assessment of the likely health impact of initiatives, usually outside of the health sector.   HIA’s are a practical tool for building health considerations into policy decisions in other sectors, i.e. through a “health in all policies” approach. 
At a basic level, the effectiveness of health interventions can be judged by extent to which they have reached their stated goals.9 Traditionally, however, systematic reviews have also looked at how studies are designed.  Not all evidence is judged to be of equal value; there are hierarchies of research design that assign different levels of usefulness of findings for the decision-making process.  
It is important to be explicit about the types of evidence that should be used for public health decision making, and how such evidence should be used.  What constitutes systematic research about health interventions?  What criteria should be used to evaluate whether evidence supports designating a practice as “effective”?   
In evaluating the quality of evidence produced by a study, reviewers may look at questions such as whether the study’s claims of success are unjustified, perhaps due to the inadequate use of controls, or whether the research design failed to identify changes that actually occurred as a result of the intervention.11  In biomedical research, such questions are best addressed through the use of randomized control trials (RCTs), the “gold standard” for evaluating the effectiveness of clinical treatments and interventions.  The USPSTF originally adopted a “hierarchy of evidence” which list a range of study designs ranked in order of decreasing internal validity (see Exhibit 1).   <set xml=true><silence msec="1500" /  > 

Exhibit 1. Hierarchy of Evidence Used by the USPSTF  <set xml=true><silence msec="1500" /  > 
 
 Category I:  Evidence from at least one properly randomized controlled trial. 
 Category II-1: Evidence from well-designed controlled trials without randomization. 
 Category II-2: Evidence from well-designed cohort or case-control analytic studies, preferably from more than one center or research group. 
 Category II-3: Evidence from multiple times series with or without intervention or dramatic results in uncontrolled experiments such as the results of the introduction of penicillin treatment in the 1940s. 
 Category III: Opinions of respected authorities, based on clinical experience, descriptive studies and case reports, or reports of expert committees.  <set xml=true><silence msec="1500" /  > 

Recognizing the limitations of a strictly hierarchical approach, the USPSTF subsequently developed a set of criteria for assessing internal validity that allows them to categorize individual studies and an overall body of evidence as good, fair or poor (see Exhibit 2).    <set xml=true><silence msec="1500" /  > 

Exhibit 2. USPSTF Criteria for grading the internal validity of individual studies  <set xml=true><silence msec="1500" /  > 
Study design: Systematic reviews Criteria:  
• Comprehensiveness of sources/search strategy used  
• Standard appraisal of included studies  
• Validity of conclusions  
• Recency and relevance  

Study design: Case-control studies Criteria:  
• Accurate ascertainment of cases  
• Nonbiased selection of cases/controls with exclusion criteria applied equally to both  
• Response rate  
• Diagnostic testing procedures applied equally to each group  
• Appropriate attention to potential confounding variables  

Study design: Randomized controlled trials (RCTs) and cohort studies Criteria: 
• Initial assembly of comparable groups: For RCTs: adequate randomization, including concealment and whether potential confounders were distributed equally among groups For cohort studies: consideration of potential confounders with either restriction or measurement for adjustment in the analysis; consideration of inception cohorts  
• Maintenance of comparable groups (includes attrition, crossovers, adherence, contamination)  
• Important differential loss to follow-up or overall high loss to follow-up  
• Measurements: equal, reliable, and valid (includes masking of outcome assessment)  
• Clear definition of interventions  
• All important outcomes considered  
• Analysis: adjustment for potential confounders for cohort studies, or intention-to-treat analysis for RCTs  

Study design: Diagnostic accuracy studies Criteria: 
• Screening test relevant, available for primary care, adequately described  
• Study uses a credible reference standard, performed regardless of test results  
• Reference standard interpreted independently of screening test  
• Handles indeterminate results in a reasonable manner  
• Spectrum of patients included in study  
• Sample size  
• Administration of reliable screening test   <set xml=true><silence msec="1500" /  > 
 
The USPSTF has also recognized the importance of external validity (i.e., how generalizable the results are to the relevant population).  Their systematic reviews now assess the internal validity (i.e., does the study provide an accurate result), external validity (i.e., is the study relevant to the setting of interest), the magnitude of effect (for the benefit of a service to be considered substantial, there must be either a small relative impact of a frequent condition with a substantial population burden, or a large impact on an infrequent condition that poses a significant burden at the individual patient level), and certainty (i.e., degree of assurance that recommended services actually produce more benefit than harm).  
The USPSTF uses tables that take into account the level of certainty and magnitude of effect (see Exhibit 3) to translate the evidence of reviews into suggestions for practice (see Exhibit 4).    <set xml=true><silence msec="1500" /  > 

Exhibit 3. USPSTF Recommendation Grid: Letter Grade of Recommendation or Statement of Insufficient Evidence Assessing Certainty and Magnitude of Net Benefit  <set xml=true><silence msec="1500" /  > 
 Grade A indicates high certainty of evidence that the magnitude of net benefits is substantial 
 
Grade B indicates moderate certainty of evidence that the magnitude of net benefits is either moderate or substantial, or high certainty of evidence that the magnitude of net benefits is moderate 
 
Grade C indicates that the certainty of the evidence is either high or moderate, or that the magnitude of net benefits is small. 
 
Grade D indicates high or moderate certainty of the evidence that the magnitude of net benefits is either zero or negative. 
 
Grade I indicates that the evidence is insufficient to determine the relationship between benefits and harms (i.e., net benefit).   <set xml=true><silence msec="1500" /  > 

Exhibit 4. What the USPSTF Grade Means and Suggestions for Practice. 
This exhibit has 3 columns labeled Grade, Definition and Suggestions for Practice.
A is defined as The USPSTF recommends the service. There is high certainty that the net benefit is substantial. And the suggestion is “Offer or provide this service.”
B is defined as The USPSTF recommends the service. There is high certainty that the net benefit is moderate or there is moderate certainty that the net benefit is moderate to substantial. And the suggestion is “Offer or provide this service.”
C is defined as The USPSTF recommends against routinely providing the service. There may be considerations that support providing the service in an individual patient. There is at least moderate certainty that the net benefit is small. The suggestion is to Offer or provide this service only if other considerations support the offering or providing the service in an individual patient.
D is defined as The USPSTF recommends against the service. There is moderate or high certainty that the service has no net benefit or that the harms outweigh the benefits. And the suggestion is to Discourage the use of this service.
I Statement is the last level of grading in the bottom row. The definition says The USPSTF concludes that the current evidence is insufficient to assess the balance of benefits and harms of the service. Evidence is lacking, of poor quality, or conflicting, and the balance of benefits and harms cannot be determined. The suggestion is to “Read the clinical considerations section of USPSTF Recommendation Statement. If the service is offered, patients should understand the uncertainty about the balance of benefits and harms.
 
There is recognition that a simple hierarchy of evidence based on internal validity provides only a limited assessment of the interventions for the community-based settings in which public health interventions occur.14  The Community Preventive Services Task Force has developed methodological criteria that are applicable to population-health settings. The Task Force assesses the body of knowledge about an intervention based on the quality of studies (suitability of design and quality of execution), number of studies, their consistency, and effect size.  Those methods have been applied to policies, health care systems, laws, education, worksite, housing, built environment, incentives, and many other types of interventions.  That information is then translated into a table that shows whether an intervention is recommended or not (See Exhibits 5 and 6 below).    <set xml=true><silence msec="1500" /  > 
 
Exhibit 5. Assessing the strength of a body of evidence on effectiveness of population-based interventions in the Guide to Community Preventive Services  <set xml=true><silence msec="1500" /  > 
Exhibit 5 has 6 columns and 4 rows. Colums are labeled vidence of effectiveness,  Execution-good or fair, Design Suitability-Greatest, moderate, or least, Number of Studies Consistent, Effect size, Expert opinion. The rows are labeled Strong, sufficient, expert opinion and insufficient. Cells across in rows have the words such as good, fair, greatest, at least 2, yes or inconsistent, sufficient or large. <set xml=true><silence msec="1500" /  > 

Exhibit 6. Relationship of strength of evidence of effectiveness and strength of recommendations  <set xml=true><silence msec="1500" /  > 
Exhibit 6 has 2 columns and 5 rows. Columns are labeled strength of evidence of effectiveness and recommendation.

 Comparative effectiveness is the application of evidence-based principles to understanding how different interventions compare to each other and answers questions such as how the benefits and harms differ overall or in specific subpopulations or situations.  The Agency for Healthcare Research and Quality (AHRQ) has used its network of Evidence-based Practice Centers (EPCs) to develop the methodology for comparative effectiveness reviews and to conduct many of them.   The term is also applied to randomized trials which compare two or more active interventions and occasionally to comparative economic evaluations.   The Institute of Medicine recognized the importance of population health in its list of national comparative effectiveness priorities.   The paucity of comparative studies of population health interventions remains a major gap
Some have proposed using well-designed evaluation studies as sources for evidence of effectiveness in public health interventions. Evaluation studies are designed to assess whether an intervention has achieved its goals, rather than seeking to control for variation among units of comparison. They observe a proximal-distal chain of events, with different levels of outcome leading to different indicators. 17 Changes in proximal indicators are more likely to be due to the direct impact of an intervention. 18   
 
To capture the multidimensional nature of public health practice, some practitioners have recently called for the development of a broader approach to assembling evidence. Measuring whether a particular intervention has succeeded in shifting the health of a population poses several challenges. As a result, the methodology for evaluating evidence in public health interventions is still being developed today; challenges and considerations for this effort are outlined below.   <set xml=true><silence msec="1500" /  > 
. What are challenges of developing a broad approach to compiling evidence for public health practice? <set xml=true><silence msec="1500" /  >  

Some have proposed using well-designed evaluation studies as sources for evidence of effectiveness in public health interventions. Evaluation studies are designed to assess whether an intervention has achieved its goals, rather than seeking to control for variation among units of comparison. They observe a proximal-distal chain of events, with different levels of outcome leading to different indicators.  Changes in proximal indicators are more likely to be due to the direct impact of an intervention.  
To capture the multidimensional nature of public health practice, some practitioners have recently called for the development of a broader approach to assembling evidence. Measuring whether a particular intervention has succeeded in shifting the health of a population poses several challenges. As a result, the methodology for evaluating evidence in public health interventions is still being developed today; challenges and considerations for this effort are outlined below. <set xml=true><silence msec="1500" /  > 
 The Multifaceted Nature of Population Health Approaches <set xml=true><silence msec="1500" /  > 

Targeting, achieving and measuring a shift in the health behaviors of a whole population is complex. Whole-population shifts require multidisciplinary and multi-sectoral strategies, with targets at various levels.  Effective programs often require multiple types of interventions with synergistic components.  Public health interventions are large-scale, long term, and concerned with external validity or “real world” applicability.   High-quality practice- or community-based research studies are a potentially important source for new information on effectiveness that can provide insights not only into whether an intervention works, but how to implement them in real-world practice.    They can also provide badly needed information on important groups, such as the disabled or those with multiple chronic illnesses, who are often excluded from efficacy studies.  These studies can also provide the important nuance about how interventions should be tailored to the context of different environments.   <set xml=true><silence msec="1500" /  > 

 Limitations of The Traditional Hierarchy of Evidence   <set xml=true><silence msec="1500" /  > 

While the clinical approaches that are addressed by EBM focus primarily on individuals, interventions in public health focus on populations.  RCTs, which use an experimental design and controls groups, are difficult to introduce to population health interventions for a variety of reasons, including expense and practical challenges.  The use of randomization to control for confounding variables is also less feasible in population-based interventions as compared to clinical trials. 20 Observational designs and other types of qualitative studies may be better-suited to describing phenomena that cannot easily be described through the use of quantitative studies. Modeling can synthesize the best available information and facilitate comparison of different strategies.  Similarly, surveillance data may provide better indicators of the success of multiple interventions than RCTs.    <set xml=true><silence msec="1500" /  > 
 The Need for Other Contextual Information   <set xml=true><silence msec="1500" /  > 

Apart from methodological rigor, other considerations must be taken into account when evaluating the effectiveness of health promotion interventions. For example, community acceptance and involvement, integration, engagement in multiple dimensions of an intervention are key indicators of success.   In general, these factors are of practical importance and need to be considered in selecting among interventions that have been shown to be effective. However, these factors alone are an inadequate rationale for choosing a good concept over one that has strong evidence of effectiveness. When novel approaches are implemented, they need to be done with strong evaluations.   However, it remains unclear how much weight should be given to non-experimental factors when evaluating an intervention.   <set xml=true><silence msec="1500" /  >  
 Measurement of Outcomes  <set xml=true><silence msec="1500" /  >  

Health and intermediate outcomes of population based interventions are measured at all points and stages of program development and implementation, and should not be limited to endpoints. The effectiveness of public health interventions should be measured at multiple points in time (e.g. proximal, distal) as health outcomes may not become evident over the short term.   <set xml=true><silence msec="1500" /  > 
What strategies are being explored to broaden the evidence-base for public health practice?  <set xml=true><silence msec="1500" /  > 
Given the challenges outlined above, there is a movement within the literature to think more broadly about how evidence is derived.  New, rigorous approaches for evaluating “best practices” and “model practices” in public health interventions are needed.  Some have called for shifting the focus in public health away from “evidence-based practice” and toward the more relevant “practice-based evidence.” One proponent of this view noted that, “as public health… strives to rise to the paradoxical challenge of evidence-based practice…the challenge is that most of the evidence is not very 
practice-based.”  Below are a few key issues to be considered in developing broadened strategies for evaluating the evidence base for public health interventions.   <set xml=true><silence msec="1500" /  > 
 Contextual Issues   <set xml=true><silence msec="1500" /  > 

Evidence based public health practice should begin with interventions of known effectiveness and an understanding of the magnitude of impact. However, choosing which to implement must take contextual factors into consideration.  Understanding the value of interventions, preferences of a communities, budget constraints and political and logistic feasibility are all relevant for making choices appropriate to the population.  The impact of some contextual factors can be studied, such as cost-effectiveness, while the effect of others can best be ascertained through deliberative processes.   Where there are no evidence based strategies, but the need for intervention is high, communities may decide to implement an intervention based on other information.  When undertaking such interventions, though, they should be subjected to careful evaluation. <set xml=true><silence msec="1500" /  >    
 Health Impact Assessment (HIA)  <set xml=true><silence msec="1500" /  > 

Health impact assessment describes a variety of methodologies to assess the health impact of proposed programs, policies or other activities.   Most often, this set of approaches is used to estimate the likely overall and distributional health effects of these interventions in non health sectors, such as education, transportation, fiscal and monetary policy, urban planning, energy, housing, commerce, agriculture.  HIA has great importance to collective efforts to improve population health because the actions in these sectors constitute very important determinants of health.  There is a rapidly growing body of literature on both methods for developing and grading evidence in HIAs as well as results of HIAs in the United States and other countries (see Resources below).     <set xml=true><silence msec="1500" /  > 
 Assessing Magnitude of Effect  <set xml=true><silence msec="1500" /  > 

While decision makers want to know what works and where, they also need to know how large an impact can be anticipated.  That information is sometimes available from evidence reviews, and, more commonly, from HIAs.   The Carter Center’s Closing the Gap project and, more recently, the National Commission on Prevention Priorities,28 have demonstrated how the magnitude of impact for specific interventions can be determined and used as part of priority-setting processes.  <set xml=true><silence msec="1500" /  > 
What resources currently exist for identifying evidence-based and best practices in public health? <set xml=true><silence msec="1500" /  >  
Several existing resources present evidence and knowledge for public health practice.  Examples of useful resources are provided below.    <set xml=true><silence msec="1500" /  > 
 The Guide to Community Preventive Services (the community guide dot org)

Now over a decade old, provides evidence-reviews and recommendations of over 200 population health interventions, with frequent additions and modifications. 

 Evidence-based Practice for Public Health (EBPH)30   <set xml=true><silence msec="1500" /  > 

Provides online access to selected evidence-based public health practices resources, knowledge domains of public health, and public health journals and databases. The resources are arranged along a pathway of evidence so that public health practitioners can easily find and use the best evidence to develop and implement effective interventions, programs, and policies. It includes various links to evidence-based guidelines, systematic reviews, filtered searches of publications and best practices. <set xml=true><silence msec="1500" /  >   
 Cochrane Public Health Group   <set xml=true><silence msec="1500" /  > 

A recent initiative that aims to undertake systematic reviews of upstream public health interventions. The demand for the Cochrane Public Health Group arose out of a call to review topics on The Cochrane Library that are outside the scope of existing Cochrane review groups which focus on the review of interventions in the medicine.32  In March 6, 2008, the Cochrane Public Health Group launched their editorial and methods meetings.33  Meeting topics and presentations included:   <set xml=true><silence msec="1500" /  > 
 Study designs for including in public health reviews  
 Study searching for public health reviews  
 Context and process evaluations on public health reviews  
 Assessing quality of studies for public health reviews  
 Including economic evaluations in public health reviews   
 Approaches to the synthesis of heterogeneous evidence   <set xml=true><silence msec="1500" /  > 

The Cochrane Public Health Group has developed a Health Promotion and Public Health Systematic Review Handbook that guides reviewers through the process of completing a systematic review that measures the effectiveness of certain public health intervention.   <set xml=true><silence msec="1500" /  > 

 National Association for City and County Health Officials (NACCHO)  Database of Model Practices in Local Public Health Agencies  <set xml=true><silence msec="1500" /  >  

A database of model practices that were identified through the application of several key criteria.  Included practices are considered to be exemplary, and replicable. 35-36  <set xml=true><silence msec="1500" /  > 
 Promising Practices Network  <set xml=true><silence msec="1500" /  > 

This resource provides summary information for programs and practices that have been proven to have a positive effect on outcomes for children and youth. Programs are rated as either “Proven” or “Promising,” based upon several considerations, including the types of outcomes affected.37 
 Health Impact Assessment:  Information & Insight for Policy38   <set xml=true><silence msec="1500" /  > 

This resource provides health impact assessment (HIA) reports as well as information about conducting and using HIA and links to related sites. <set xml=true><silence msec="3000" /  > 
APPENDIX 1 
Ad Hoc Group on Evidence-based Practices 
In June of 2009, the U.S. Department of Health and Human Services requested that the Secretary’s Advisory Committee provide feedback on what criteria should be used to select “evidence-based” or “knowledge-based” actions for inclusion in Healthy People 2020.  The Committee convened an Ad Hoc Group to provide recommendations on these issues.  The Ad Hoc group met once, on June 9, 2009, to inform the development of recommendations. Key members of the group substantially revised an earlier draft of this document that had been produced by the National Opinion Research Center.  The document was then reviewed, edited, and approved by a vote of the full Secretary’s Advisory Committee.  It summarizes expert ideas on evidence-based practice in public health, challenges in assessing evidence in support of public health interventions, and existing resources for evidence-based public health practices.  
 
Bruce Nedrow (Ned) Calonge, MD, MPH. 
Colorado Department of Public Health and Environment 
 
J. Michael McGinnis, MD, MPP 
Institute of Medicine 
 
Tracy Orleans, PhD 
Robert Wood Johnson Foundation  
 
Steven M.  Teutsch, MD, MPH  
Los Angeles County Dept. of Public Health 
 
Jonathan E. Fielding, MD, MPH, MA, MBA 
Los Angeles County Dept. of Public Health  
 
Russ Glasgow, PhD 
Kaiser Permanente 
 This concludes the report on evidence based actions in public health.