The first step in critical appraisal – deciding what design your research study uses.

Critical appraisal has been made much easier by the development of tools that guide you through the process of what aspects of the research study you should be evaluating.  Some argue that it isn’t until you become experienced in research methods that you can properly evaluate the quality of the methods used by others.  I think this is a valid point but we all have to start somewhere, and critical appraisal is one of the best ways I know to learn about research methods if you aren’t able to actively do a research project.

The quality of a research study is determined by how appropriate the approach, the design and the methods used were to address the question.  The first decision we make when evaluating a study therefore is whether this question would be best answered using a qualitative approach (using interviews or focus groups to find out about people’s experiences) or a quantitative approach (collecting data using questionnaires and measurement tools).  Some questions are best answered by using a combination of both approaches (mixed methods or multi-methods).  Quite often the person undertaking the critical review will decide ahead of time (a priori) which approach is appropriate and use that as an inclusion criterion for the retrieval of papers.

Once you can determine the approach a study has taken you move onto deciding what design the researchers have used.  As a novice this can be particularly difficult.  If you are lucky the researchers have included the design in the abstract of their paper. If not, you will have to work it out by looking at whether there is a hypothesis or not (usually a sign of a quantitative analytical study) or an aim (usually a sign of an observational study).

Here are some notes about different types of studies, with thanks to Josette Bettany-Saltikov from whose book ‘How to do a systematic literature review in nursing‘ I have adapted the diagrams.

Quantitative designs

Case reports and case series

A report of the treatment of a single patient or more than one patient. Usually published because the condition or treatment is very rare.

Case report

Case control studies

Patients who have a condition are compared with those who don’t.  The data for the study is taken from the medical records and patient recall (memory).

Case control studies

Cohort studies

This design follows a large group of people for a long time who all have the condition of interest, usually as they receive a treatment, or to observe how the condition changes.

Cohort studies

Randomised controlled trials

Used to explore the effect of treatments by comparing a group who are given the treatment with one who are not.  The strength of the design is that by randomly allocating patients to the active and control groups you will end up with broadly similar people in each group (in terms of the variables that could inadvertently influence the outcome being measured).

RCT

Qualitative designs

Phenomenology

Exploring the lived experience of a group of patients or staff.

Ethnography

Describing a culture – usually by becoming part of that group to understand it.  Good to understand how people see their own experiences.

Grounded theory

The researcher aims to develop a theory that explains events and behaviours.

The other aspect to design is to determine when the data was collected and whether it was taken from a source that already existed (e.g. the patients’ notes) or whether it was collected for the study.

  • At one point in time = cross-sectional
  • At several time points = longitudinal
  • Data that already existed and was collected for a different purpose = retrospective
  • Data is being collected for the research study = prospective.

Once you have a basic idea of what types of study there can be you can usually work out what the design is even if the authors don’t state it.

You might find this decision tree for research design helpful.  Feedback would be very helpful to improve the design of this resource.

Research design flow diagram

2 thoughts on “The first step in critical appraisal – deciding what design your research study uses.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s