Updating Evidence Summaries
These questions and answers were developed from our Scientific Writers forums held in February 2022. Scientific Writer forums will be held each year, and this library of Q & As will be added to as topics arise.
Clinical Bottom Line
Q: Do I have to review the whole evidence summary or just look for new evidence?
As the updater of the evidence summary, you take responsibility for the whole summary. Read the question carefully and make sure the evidence included addresses the clinical question. Reviewing the summary involves correcting any previous grammatical errors, or reworking existing evidence points, removing existing evidence where appropriate (eg, superseded my higher quality evidence), in addition to adding the newly found evidence.
Q: When adding new evidence should I write in my own words and should I include data – what is the best?
The evidence points must be written in your own words; but data can be quoted directly as that is something that doesn’t change. Only relevant data that supports the evidence statement should be included.
Q: What type of data do I include, for example p values, odd ratios and relative risk and more complex statistics?
We’ve moved away from reporting these more complex statistics in the evidence summary and report the findings narratively. Keep in mind who are the end users for the resources (health professionals that may have a limited understanding of complex statistics). If you frame the statistics in a sentence, then they’ll understand the impact of the finding on their practice. For example: a significant increase in {outcome} due to the intervention.
Q: What type of information should I include when describing a study in an evidence point?
When summarising the evidence, it is important to describe and report key information from the paper. As a guide, this should include the purpose of the study; if there was an intervention, include a short description of what this involved and a summary of the key findings. For example, rather than just saying education was provided. What did the education look like? Was it a one-hour group session, 30 minutes or a one-on-one session? It is also important to report on the quality and any limitations of the findings. Well reported evidence-based guidelines will often report on the level of confidence with the evidence eg, a strong recommendation based on moderate quality evidence.
Characteristics of the Evidence
Q: What information do I need to include to describe the included evidence?
We want enough information here to provide a good summary of what the study is. If it’s a systematic review, then we need to report the included study designs. This is important as it can determine the level of evidence for the systematic review. If possible, include the number of participants, and if relevant, a description of them. ie: children, aged 1-5 years.
Best practice Recommendations
Q: How do I know when to make changes to the Best Practice Recommendations?
It’s important that the recommendations presented in the summary strongly align with the evidence. The following questions can assist with reviewing and updating this section:
Does the new evidence you’ve added change an existing recommendation? If so, the recommendation should be amended to reflect the change to practice.
Does the new evidence provide extra support to the recommendation, in which case does the grade change?
Does the new evidence added warrant a new recommendation for practice to be developed? If so, develop a new recommendation for practice.
It can feel a little daunting making a change to developing a new recommendation for practice. That’s why we ask you to use track changes and comments so we can support or advise you on any suggested changes you make.
Searching
Q: Should the search strings be placed in the technical development report?
This can be very helpful. We are aware there are a variety of ways scientific writers currently present their search strategy, and we are happy to accept the variations. What we’re looking for is to make sure all the keywords and search terms are captured so nothing obvious is missed. Standardisation in the reporting is something our team will discuss to have consistency going forward, mindful that different databases have different search requirements.
Q: What type of evidence should I include? I usually try to only include systematic reviews, randomized controlled trials and clinical practice guidelines.
The type of evidence included should relate to the topic and clinical question and how much evidence there is. Ideally, we want evidence summaries to be based on high-level, high-quality evidence like systematic reviews, and evidence-based clinical guidelines. However, not every topic has these types of evidence, so we need to work our way down the evidence pyramid and include the best available evidence. We try to refrain from including literature reviews and scoping reviews, often because a comprehensive search or critical appraisal has not been conducted. In such cases it is better to evaluate the primary studies for inclusion.
Including studies and referencing
Q: Do I remove a study just because it is old?
No. We want studies that are no longer relevant to be removed. But just because it is old – it doesn’t mean it must be removed – there is no set time limit – it still might be the best available evidence to answer the question. If you are removing a study, it is useful to leave a short comment as to why. For example, you might remove an RCT because you have added a systematic review that includes the RCT. Another example is you’ve added a new systematic review that supersedes the previous systematic review that was there.
Q: Do we add in newer evidence that reports the same findings as already included higher level studies?
No, it is not necessary to add lower-level evidence if there is already high-level evidence that answers the clinical question.
Q: Do I need to review the existing references?
Yes. It’s a great idea to read through the existing references. It is important to check if an included systematic review has been updated (eg, Cochrane reviews, will quite often have updates. We add the Cochrane database number to the reference so it’s easy to check if there’s been an update or not. Also, clinical guidelines can be updated, and often contain a weblink. Check the link to see if the guidelines are still accessible and referencing the latest version.
Important to note: If you do remove references, check to make sure if it is included in the introduction paragraph as this paragraph will need to be reworded and referenced.
Q: What if I’ve found an RCT that was published prior to a systematic review, but wasn’t captured in the systematic review – do I include it?
It’s difficult to answer this specifically as it depends on the reason why the RCT was excluded from the systematic review. It may be excluded due to poor quality, or it may not have been aligned to the specific PICO or the inclusion criteria of the systematic review. Sometimes, the evidence summary question is broad, and systematic review questions are very specific, so in some instances you could have systematic reviews and other intervention studies that still address the evidence summary question.
If the additional RCT has a slightly different focus then it can be useful to include, but if it has the exact same focus and was published before the systematic review – I would include the systematic review.
Overall, it is important that the evidence we are including answers the clinical question. Sometimes that will mean we have multiple systematic reviews and RCTs because they are addressing slightly different components of that question.
Q: How do I report conflicting evidence?
We need to present the best available evidence, high-quality evidence, which also involves looking at the critical appraisal scores when deciding what to include. That means preferring high quality and moderate-high quality over low quality evidence. If you still have conflicting evidence, it is very reasonable to provide both, as we provide the best available evidence. When it comes to present the best practice recommendations, we factor in the conflicting evidence.
Other
Q: I note there are discount for staff on training courses, are these discounts available for scientific writers?
That’s an excellent question, and something we will definitely look at going forward, but that’s something that’s not available at the moment.
Q: Can I get access to the Evidence Summaries that I update and see the other resources?
A: Our database is hosted by Wolters Kluwer (WK) and is membership based. Unfortunately, we cannot provide access to the database to our scientific writers due to our commercial agreement with WK. Your institution may subscribe to the JBI EBP database, and if so, you can access via their portal.
Q: Do I get notified when a summary I updated has been published or cited?
A: Unfortunately, no. Being published in the JBI EBP database is a little different to being published in a journal as we are a membership only database providing point-of-care resources. The only times you would see something that relates back or cites back to an evidence summary is if somebody else has conducted a piece of research and used our evidence summaries to inform that practice. This happens quite often with implementation projects.
Q: How do you create an Evidence Summary from the beginning?
Creating an original evidence summary is possible with our Field Collaborator program. The JBI Field Editor together with the Expert Reference Group examine the field’s taxonomy and identify topics for development. A PICO framed question is formed and then allocated to a Field Collaborator who develops the evidence summary in collaboration with the Field Editor. It is possible to be both a Scientific Writer and a Field Collaborator. See our field collaborator webpage.