July 2, 2016
The Limits of Randomized Controlled Trials: Who Agrees to be Recruited?
Randomised controlled trials (RCTs) of interventions are a gold standard source of evidence in medicine, as people like Ben Goldacre have argued repeatedly. As people are allocated to receiving the intervention at random, this should eliminate many of the biases that come from people self-selecting for interventions they would like.
But RCTs are vulnerable to another sort of bias – that of deciding whether to take part in the trial at all. The study I am discussing here, by Rogers and collaborators, takes a very thorough look at why older people decided to take part in a trial that tested an intervention which is designed to get older people to walk more.
Study participants were recruited from three general practices in relatively affluent parts of England, Oxfordshire and Berkshire. Potential participants were identified from the general practitioners’ records. General practitioners then filtered out those whom they deemed unsuitable for the intervention, and then the invitations to the remaining people were sent out through the practices.
This means that while the researchers did not see the names and addresses of the non-participants, they still had access to some basic demographic information which allowed them to compare who did and did not show interest in the trial. This information included age, gender, whether they were invited on their own or as a couple, and the socioeconomic deprivation index of the area they lived in – but not the area itself.
988 people were contacted initially. Everyone had three options: to take part in the trial, to complete a survey with information about why they chose not to take part, and not to respond at all. 298 (30.2% or one in three) people agreed to participate, and 690 were not interested. Of those 690, 183 (26.5% or one in four) returned the survey, and 77 of the 183 (42%) agreed to be contacted further about their reasons for not participating. Rogers then interviewed 15 of these people herself; the interviews stopped after 15 because no new insights emerged.
Instead of discussing the complex pattern of results that emerged from the study, I would like to highlight two findings that I consider to be the most interesting.
Finding 1: The people who don’t respond at all are very different from the people who will return your non-participation survey.
Table 1 of the paper shows the overall demographic differences between participants and non-participants, while Table 2 looks at the demographic differences between participants and non-participants that returned the survey. The pattern that emerges from Table 1 is that people are less likely to take part if they are male and if they live in a deprived area. Age and whether they were invited as a couple or not did no matter. Table 2, on the other hand, shows no difference at all on any of these four metrics.
Finding 2: Taking part in a trial is hard work for participants.
While the most common reason people cited for not taking part was that they were already physically active (67.3% of the 183 who returned the non-participation survey), the second most important reason was that they just didn’t have the time (44%).
The qualitative interviews provide an insight into the demands that taking part in the trial would place on participants. They would have to
- find time in lives that were already full of family commitments and activities
- stay with the trial for three months
- walk regularly in the dark winter
- look after an accelerometer device to measure physical activity
- walk regardless of other health issues such as chronic pain, depression, or knee problems
- change their existing habits and routines
Conclusion
The people who ended up taking part in the trial were not only more wealthy and more likely to be female, but also more likely to be able to organize their lives around increased physical activity.
What does that mean for clinical practice? While it appears to be very easy to tell people to just be more active, the recruitment patterns for this trial indicate that those who might need help the most don’t necessarily contribute to the evidence base that doctors are told to rely on.
Rogers, A., Harris, T., Victor, C., Woodcock, A., Limb, E., Kerry, S., … Cook, D. G. (2014). Which older people decline participation in a primary care trial of physical activity and why: insights from a mixed methods approach. BMC Geriatrics, 14(1), 46. http://doi.org/10.1186/1471-2318-14-46
How Can We Design Assistive Technology that Helps People with Early-Onset Dementia?
Living with dementia can be hard for the person with dementia and for the people who care for them. Good support can make life a lot easier, and create space for moments of contentment, joy, and happiness.
In the past decades, assistive technology developers have sought to provide part of this support through specialized technology. Just as a prosthetic leg can help amputees walk, developers have created prosthetic memory solutions that fill the gaps in a person’s own memory.
However, finding the package of solutions and services that is best for the person with dementia and their carers can be very difficult. Often, these services are put together by occupational therapists.
In the paper I am discussing here, three occupational therapists, academics and practitioners from the North of Norway, Cathrine Arntzen, Torhild Holte, and Rita Jentoft, reflect on the process of creating a technology solution that works. They focus on people who are diagnosed with dementia relatively early in their lives, in their fifties and sixties. These are baby boomers who tend to be far more familiar with technology than people in their eighties and nineties, who are more likely to be diagnosed with dementia.
To give you an idea of the complexity of these solutions, the assistive technology introduced, which is documented in Table 1 of the paper, ranged from a simple digital calendars to a package of four devices, and included specialized solutions such as a reminder connected to the timer of the coffee machine and non-technological solutions such as whiteboards.
Methodology
For their study, they used a longitudinal, qualitative design. This means that they followed 12 people and their carers for a year after the first home visit, when their needs were assessed. They talked to their participants every three months, reviewing issues that had come up at past interviews, and exploring new issues that had arisen. They also took notes of their own observations.
Qualitative data such as interviews are notoriously difficult to analyse. Each analyst approaches the text with their own preconceptions and ideas. Therefore, texts are often analysed by two to three people over several iterations, comparing and contrasting their findings, to ensure that their interpretation is grounded in what the participants told them.
As a result of analysis, patterns and themes emerge as well as individual experiences that highlight wider issues. There are many ways to ensure that these findings can be useful in different contexts. In this study, Arntzen, Holthe, and Jentoft interpreted their findings in light of a particular theory of lived experience, phenomenology.
Findings
Arntzen, Holte, and Jentoft identified five elements of a successful assistive technology. As I list them, I will comment on each from my own experience of working in this field.
1 The technology has to address an actual need, which can be practical, emotional, or about the way people relate to each other.
Comment: This means that careful initial assessment is important, and default packages are likely to fail.
2 The technology has to fit in with people’s established habits and problem solving strategies, because they reflect how a person thinks about and relates to the world.
Comment: If a piece of assistive technology is introduced because it provides useful data to central services, even though it would require people with dementia and their carers to rethink the way they organize their lives, it is highly likely to fail.
3 The technology needs to be reliable and trustworthy, and people need to feel good about it.
Comment: This means that assistive technology needs to be well designed and tested, requiring high quality software engineering, and supported by qualified engineers who can intervene quickly in case of malfunctions.
4 The technology needs to be user friendly, adaptable, and easy to manage.
Comment: Ideally, all technology would be like that, but all too often, technology is designed primarily to provide data and to enforce standard procedures, and to enforce a predictable life with no room for spontaneity. This has been a problem for a long time, and it is a typical of the conflicts between stakeholders (people with dementia who want to live their own lives as they used to; carers; social care; health care; policy)
5 The technology needs to interest and engage the family carer. It is likely that the family carer will be the one to look after the technology, keep it up to date, make sure it works, and use the more complex functionalities. If they like it, and if it engages them, it is more likely to be used. A case in point was the digital calendar, which proved very popular with the carers, less so with the people with dementia.
Comment: Family carers are sometimes overlooked in the work on supporting people with dementia to live in the community, as there is a strong emphasis on helping those who do not have family live independently – those are the people who require more social care time. Carers are also often assumed to be children, and spouses are assumed to be technophobic, as they are older. However, all of the people with dementia in this study were cared for by their spouse, who was in a similar age group. This means that we need to make sure technology for people with dementia is also accessible to older people without cognitive impairment.
Conclusion
While much of what Arnzten, Holthe, and Jentoft found in their paper will not be new to people who work in the field, I still think that their paper is a salutary reminder of just how important adaptable, flexible technology solutions are. Fixed packages of standard technology may be easier to maintain and to prescribe, but will they pay for themselves in actual daily use? If Arntzen, Holthe, and Jantoft are right, then this is highly unlikely.
Arntzen, C., Holthe, T., & Jentoft, R. (2016). Tracing the successful incorporation of assistive technology into everyday life for younger people with dementia and family carers. Dementia, 15(4), 646–662. http://doi.org/10.1177/1471301214532263
Speech and Science 