131:(ISPOR) has issued guidelines on establishing an equivalence between modes of administration. Their approach is hierarchical and depends on the degree of change made during the process of migration from paper to electronic format. Three levels are suggested. At the lowest level, where the least change has been made, cognitive interviewing of patients as a check that they construe ePRO and paper, in the same way, is sufficient. This level includes both trivial changes (touch rather than circle a response choice) as well as changes that are supported by empirical findings in the literature. At the second level, equivalence studies comparing the scores obtained from the two modes should be carried out. At the third level, where most change has occurred, the ePRO instrument must be treated as a new instrument, and complete psychometric validation carried out.
81:
unduly influenced by how the patient is feeling at the time of making the rating, and by maximum severity rather than average severity during the assessment interval. Diaries can overcome this problem by recording severity either on a momentary basis ("How bad is your pain right now?") or by recalling over short periods ("How bad has your pain been today?"). However, when diary data is collected on paper, it is not known when the ratings are actually made, and there is evidence that compliance may be quite poor. In one study, patients were given an instrumented paper diary that recorded covertly when it was opened. The study showed frequent cases of "back-filling", filling in a batch of entries sometimes, often days, after they were due, and even in some cases of "forward filling", completing entries before they were due.
85:
ePRO applications typically achieve compliance rates of over 80%, often over 90%. Electronic diaries also have benefits in that they only allow valid, in-range entries to be made. Devices such as PDAs allow reminders to be given to patients when entries are due. Systems generally transfer data promptly to a central server, allowing tailored feedback to be given to patients. This can also improve compliance. Electronic diaries also eliminate the need for manual editing and entry of data, time-consuming and error-prone processes.
69:). The user calls into a dedicated phone line, and hears a spoken script that details the question, and the possible responses. Each response option is given a number, and the user presses the corresponding number key on the phone keypad to record the choice. IVR systems are more often used for diaries, with the patient phoning in e.g. from home, but they can be used in a clinic setting.
59:
clinical setting. Computers generally run dedicated ePRO applications - the use of the web for ePRO is not yet widespread. Typically a single question at a time is presented on the screen, with a set of possible response options. The user taps on the appropriate response with a finger or stylus, then moves on to the next questions.
111:
From the early years of ePRO there has been concern about whether all patients can cope with computer technology. This is important, as if significant numbers of patients refuse to take part in clinical trials because of dislike of computers then there will be bias in the study population. One of the
84:
Electronic diaries automatically time-stamp all entries and can be set up to only allow entry within specified time windows. This improves compliance and ensures that true compliance can be documented. Documenting compliance is important if ePRO data are to be used to support regulatory applications.
124:
Establishing the validity of an ePRO instrument is in principle no different from that for other methods, such as paper. However most of the instruments in current use have been validated in paper form, and we need to ask (1) whether the paper data can be used to establish validity of the electronic
80:
Diaries are used when it is desirable to obtain frequent assessments over a period of time, for example when a condition fluctuates in severity. In such cases recall of severity over a period of time is unlikely to be accurate. Research has shown substantial bias in such summary recall, with ratings
203:
As well as clinical trial use, ePRO methods may be used to support patients in regular care. An example of this is the collection of symptom data from patients undergoing chemotherapy, using handheld diaries. This allows clinic staff to monitor outpatients, and to identify the occurrence of adverse
98:
and activities of daily living, for example, in some detail. They more often include branching logic ("if YES continue with the next question, if NO, go to question 34"). Such branching logic can be handled automatically by the ePRO application, and it is often not necessary for the patient even to
138:
It is not always necessary to validate an ePRO measure against a pre-existing paper version. In some cases, an instrument may be developed and validated from the beginning in electronic form. More commonly, perhaps, new instruments will be developed in parallel for paper and electronic use, as is
134:
There is a great deal of evidence supporting the general equivalence of paper and ePRO methods. Gwaltney and colleagues have reported a meta-analysis in which they included 46 studies evaluating 278 scales. They concluded that there was good agreement between paper and ePRO, and no evidence of
58:
are most often touch-screen devices, ranging from wearables, hand-held devices and smartphones, through iPads and other Tablet computers. The smaller devices are often used as electronic diaries, designed to be used for symptom reporting on a daily basis. Larger devices are generally used in a
93:
The other main setting for ePRO is the clinic, with questionnaires completed when patients come in for their scheduled visits. In this supervised situation, compliance is less of an issue. The questionnaires used in site-based ePRO are often longer and more complex than those used in diaries,
102:
As with electronic diaries, the prevention of out-of-range or inconsistent entries, and the elimination of manual editing and data entry are important features of site-based ePRO. Missing data within a questionnaire can be reduced or eliminated, and this is important as missing data has been
115:
Elderly patients, and those not familiar with computers, might be expected to have more problems. But these groups also show high acceptance of ePRO, and again often prefer it to paper. Thus there seems to be no great barrier to recruiting representative patient samples in ePRO studies.
142:
Standards of validity must be maintained throughout every target language population. In order to ensure that developmental standards are consistent in translated versions of an ePRO instrument, the translated instrument undergoes a process known as
535:
Richter JG, Becker A, Koch T, Nixdorf M, Willers R, Monser R, Schacher B, Alten R, Specker C, Schneider M (2008). "Self-assessments of patients via Tablet PC in routine patient care: comparison with standardised paper questionnaires".
147:
in which the preliminary translation is adapted to reflect cultural and linguistic differences between diverse target populations. This process also accounts for any formatting errors that may occur in languages using non-Roman fonts.
35:
that is collected by electronic methods. ePRO methods are most commonly used in clinical trials, but they are also used elsewhere in health care. As a function of the regulatory process, a majority of ePRO questionnaires undergo the
221:
Stone AA, Broderick JE, Shiffman SS, Schwartz JE (2004). "Understanding recall of weekly pain from a momentary assessment perspective: absolute agreement, between- and within-person consistency, and judged change in weekly pain".
429:
Bernhard J, Cella DF, Coates AS, Fallowfield L, Ganz PA, Moinpour CM, Mosconi P, Osoba D, Simes J, Hurny C (1998-03-15). "Missing quality of life data in cancer clinical trials: serious problems and challenges".
112:
earliest ePRO studies used a LINC-2 minicomputer to collect patient data. The majority of patients preferred the computer to paper data collection. Similar findings have been reported from many later studies.
657:
Tiplady B, Goodman K, Cummings G, Lyle D, Carrington R, Battersby C, Ralston S (2010). "Patient
Reported Outcomes in Rheumatoid Arthritis: Assessing the equivalence of electronic and paper data collection".
695:"Recommendations on Evidence Needed to Support Measurement Equivalence between Electronic and Paper-based Patient-Reported Outcome (PRO) Measures: ISPOR ePRO Good Research Practices Task Force Report"
622:
Allenby A, Matthews J, Beresford J, McLachlan SA (2002). "The application of computer touch-screen technology in screening for psychosocial distress in an ambulatory oncology setting".
135:
systematic bias. This general finding, of course, does not guarantee that any specific migration will lead to equivalence, and each case must be reviewed and documented.
200:. In the case of estradiol/levonorgestrel, detailed ePRO data on bleeding/spotting from a one-year clinical trial are presented in the patient information leaflet.
443:
954:
940:
925:
854:
386:
Friedberg F, Sohl SJ (2008). "Memory for
Fatigue in Chronic Fatigue Syndrome: The Relation Between Weekly Recall and Momentary Ratings".
314:
855:"Evaluation of a mobile phone-based, advanced symptom management system (ASyMS-¬) in the management of chemotherapy-related toxicity"
992:
853:
Kearney, N.; McCann, L.; Norrie, J.; Taylor, L.; Gray, P.; Gee-Lennon, M.; Sage, M.; Miller, M. & Maguire, R. (2009-01-04).
736:"Equivalence of Electronic and Paper-and-Pencil Administration of Patient-Reported Outcome Measures: A Meta-Analytic Review"
500:
Crawley JA, Kleinman L, Dominitz J (2000). "User preferences for computer administration of quality of life instruments".
579:
Yarnold PR, Stewart MJ, Stille FC, Martin GJ (1996). "Assessing functional status of elderly adults via microcomputer".
322:
997:
339:"Randomized, Cross-Over Evaluation of Mobile Phone vs Paper Diary in Subjects with Mild to Moderate Persistent Asthma"
40:
process. When the data is captured for a clinical trial, the data is considered a form of
Electronic Source Data.
1002:
987:
32:
777:"Progress in Assessing Physical Function in Arthritis: PROMIS Short Forms and Computerized Adaptive Testing"
181:
17:
72:
There are also a number of custom devices designed specifically for use as ePRO data collection devices.
839:
693:
Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, Lenderking WR, Cella D, Basch E (2009).
144:
37:
465:
Slack WV, Hicks GP, Reed CE, Van Cura LJ (1966-01-27). "A computer-based medical-history system".
315:"Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims"
895:
806:
675:
604:
561:
517:
411:
247:
139:
the case with the PROMIS (Patient-Reported
Outcomes Measurement Information System) initiative.
950:
936:
933:
Electronic
Patient-reported Outcome Measures: An implementation handbook for clinical research
921:
887:
798:
757:
716:
639:
596:
553:
482:
447:
403:
368:
296:
239:
125:
version, and (2) whether data from paper and electronic versions can be used interchangeably.
877:
869:
788:
747:
706:
667:
631:
588:
545:
509:
474:
439:
395:
358:
350:
286:
278:
231:
189:
95:
363:
338:
185:
156:
Several successful regulatory approvals have used ePRO data in recent years, including
291:
266:
981:
752:
735:
711:
694:
671:
635:
415:
197:
810:
679:
608:
565:
521:
251:
899:
177:
99:
know that branching is taking place. This makes it easier for the patient to use.
444:
10.1002/(sici)1097-0258(19980315/15)17:5/7<517::aid-sim799>3.3.co;2-j
478:
354:
193:
173:
165:
49:
513:
235:
873:
592:
282:
549:
157:
891:
802:
761:
720:
643:
557:
407:
372:
300:
243:
793:
776:
600:
486:
451:
265:
Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR (2002-05-18).
169:
972:
399:
48:
The two main methods currently used for ePRO are computers (including
775:
Fries, J.; Cella, D.; Rose, M.; Krishnan, E. & Bruce, B. (2009).
161:
129:
The
International Society for Pharmacoeconomics and Outcomes Research
882:
968:
International
Society for Pharmacoeconomomics and Outcomes Research
824:
McCallister E, Usdin S (5 December 2011). "A PROfessional Trial".
128:
66:
103:
identified as a crucial quality issue in questionnaire data.
973:
Patient-Reported
Outcomes Measurement Information System
967:
945:
Stone AA, Shiffman S, Atienza AA, Nebeling L (eds.)
918:
ePRO: Electronic
Solutions for Patient-Reported Data
65:
normally use an interactive voice response system (
8:
734:Gwaltney CJ, Shields AL, Shiffman S (2008).
388:International Journal of Behavioral Medicine
267:"Patient non-compliance with paper diaries"
881:
792:
751:
710:
362:
290:
204:reactions that may require intervention.
949:(2007) Oxford: Oxford University Press.
337:Meltzer EO, Kelley N, Hovell MF (2008).
213:
947:The Science of Real-Time Data Capture
7:
25:electronic patient-reported outcome
14:
753:10.1111/j.1524-4733.2007.00231.x
712:10.1111/j.1524-4733.2008.00470.x
672:10.2165/11535590-000000000-00000
636:10.1046/j.1365-2354.2002.00310.x
538:Annals of the Rheumatic Diseases
624:European Journal of Cancer Care
467:New England Journal of Medicine
1:
323:Food and Drug Administration
781:The Journal of Rheumatology
581:Perceptual and Motor Skills
479:10.1056/NEJM196601272740406
355:10.2174/1874306400802010072
1019:
916:Byrom B, Tiplady B (eds.)
514:10.1177/009286150003400119
236:10.1016/j.pain.2003.09.020
15:
874:10.1007/s00520-008-0515-0
862:Supportive Care in Cancer
593:10.2466/pms.1996.82.2.689
283:10.1136/bmj.324.7347.1193
52:) and telephone systems.
993:Clinical data management
502:Drug Information Journal
120:Validity and equivalence
33:patient-reported outcome
920:(2010) Farnham: Gower.
550:10.1136/ard.2008.090209
271:British Medical Journal
16:For uses of E-Pro, see
931:Byrom B, Mühlhausen W
432:Statistics in Medicine
18:E-Pro (disambiguation)
794:10.3899/jrheum.090358
145:Linguistic validation
107:Patient acceptability
38:linguistic validation
343:Open Respir. Med. J
277:(7347): 1193–1194.
998:Evaluation methods
400:10.1007/bf03003071
76:Electronic diaries
955:978-0-19-517871-5
941:978-1-72028-110-8
926:978-0-566-08771-4
544:(12): 1739–1741.
1010:
904:
903:
885:
859:
850:
844:
843:
836:
830:
829:
821:
815:
814:
796:
787:(9): 2061–2066.
772:
766:
765:
755:
731:
725:
724:
714:
690:
684:
683:
654:
648:
647:
619:
613:
612:
576:
570:
569:
532:
526:
525:
497:
491:
490:
462:
456:
455:
438:(5–7): 517–532.
426:
420:
419:
383:
377:
376:
366:
334:
328:
327:
319:
311:
305:
304:
294:
262:
256:
255:
218:
1018:
1017:
1013:
1012:
1011:
1009:
1008:
1007:
1003:Quality of life
988:Clinical trials
978:
977:
964:
913:
911:Further reading
908:
907:
857:
852:
851:
847:
838:
837:
833:
823:
822:
818:
774:
773:
769:
740:Value in Health
733:
732:
728:
699:Value in Health
692:
691:
687:
656:
655:
651:
621:
620:
616:
578:
577:
573:
534:
533:
529:
499:
498:
494:
464:
463:
459:
428:
427:
423:
385:
384:
380:
336:
335:
331:
317:
313:
312:
308:
264:
263:
259:
220:
219:
215:
210:
190:post-menopausal
154:
122:
109:
96:quality of life
91:
78:
46:
21:
12:
11:
5:
1016:
1014:
1006:
1005:
1000:
995:
990:
980:
979:
976:
975:
970:
963:
962:External links
960:
959:
958:
943:
929:
912:
909:
906:
905:
868:(4): 437–444.
845:
831:
816:
767:
746:(2): 322–333.
726:
705:(4): 419–429.
685:
666:(3): 133–143.
649:
630:(4): 245–253.
614:
587:(2): 689–690.
571:
527:
492:
473:(4): 194–198.
457:
421:
378:
329:
326:. 15 May 2020.
306:
257:
230:(1–2): 61–69.
212:
211:
209:
206:
192:symptoms, and
186:levonorgestrel
153:
150:
121:
118:
108:
105:
90:
87:
77:
74:
45:
42:
13:
10:
9:
6:
4:
3:
2:
1015:
1004:
1001:
999:
996:
994:
991:
989:
986:
985:
983:
974:
971:
969:
966:
965:
961:
956:
952:
948:
944:
942:
938:
934:
930:
927:
923:
919:
915:
914:
910:
901:
897:
893:
889:
884:
879:
875:
871:
867:
863:
856:
849:
846:
841:
835:
832:
827:
820:
817:
812:
808:
804:
800:
795:
790:
786:
782:
778:
771:
768:
763:
759:
754:
749:
745:
741:
737:
730:
727:
722:
718:
713:
708:
704:
700:
696:
689:
686:
681:
677:
673:
669:
665:
661:
653:
650:
645:
641:
637:
633:
629:
625:
618:
615:
610:
606:
602:
598:
594:
590:
586:
582:
575:
572:
567:
563:
559:
555:
551:
547:
543:
539:
531:
528:
523:
519:
515:
511:
507:
503:
496:
493:
488:
484:
480:
476:
472:
468:
461:
458:
453:
449:
445:
441:
437:
433:
425:
422:
417:
413:
409:
405:
401:
397:
393:
389:
382:
379:
374:
370:
365:
360:
356:
352:
348:
344:
340:
333:
330:
325:
324:
316:
310:
307:
302:
298:
293:
288:
284:
280:
276:
272:
268:
261:
258:
253:
249:
245:
241:
237:
233:
229:
225:
217:
214:
207:
205:
201:
199:
198:myelofibrosis
195:
191:
187:
183:
179:
175:
171:
167:
163:
159:
151:
149:
146:
140:
136:
132:
130:
126:
119:
117:
113:
106:
104:
100:
97:
88:
86:
82:
75:
73:
70:
68:
64:
60:
57:
53:
51:
43:
41:
39:
34:
30:
26:
19:
946:
932:
917:
865:
861:
848:
834:
825:
819:
784:
780:
770:
743:
739:
729:
702:
698:
688:
663:
659:
652:
627:
623:
617:
584:
580:
574:
541:
537:
530:
505:
501:
495:
470:
466:
460:
435:
431:
424:
394:(1): 29–33.
391:
387:
381:
346:
342:
332:
321:
309:
274:
270:
260:
227:
223:
216:
202:
178:fibromyalgia
155:
141:
137:
133:
127:
123:
114:
110:
101:
92:
83:
79:
71:
62:
61:
55:
54:
47:
28:
24:
22:
660:The Patient
508:: 137–144.
194:ruxolitinib
174:milnacipran
166:eszopiclone
152:In practice
50:smartphones
982:Categories
840:"DailyMed"
826:BioCentury
208:References
94:assessing
89:Site-based
63:Telephones
416:144865647
349:: 72–79.
182:estradiol
158:ketorolac
56:Computers
892:18953579
883:1893/845
811:22351061
803:19738214
762:18380645
721:19900250
680:74444503
644:12492461
609:36252525
566:45468097
558:18647853
522:57179467
408:18444018
373:19412327
301:12016186
252:34037052
244:14715390
170:insomnia
935:(2018)
900:9840362
601:8724947
487:5902618
452:9549801
364:2606655
44:Methods
31:) is a
953:
939:
924:
898:
890:
809:
801:
760:
719:
678:
642:
607:
599:
564:
556:
520:
485:
450:
414:
406:
371:
361:
299:
292:111114
289:
250:
242:
164:pain,
162:ocular
896:S2CID
858:(PDF)
807:S2CID
676:S2CID
605:S2CID
562:S2CID
518:S2CID
412:S2CID
318:(PDF)
248:S2CID
951:ISBN
937:ISBN
922:ISBN
888:PMID
799:PMID
758:PMID
717:PMID
640:PMID
597:PMID
554:PMID
483:PMID
448:PMID
404:PMID
369:PMID
297:PMID
240:PMID
224:Pain
196:for
188:for
176:for
168:for
160:for
29:ePRO
878:hdl
870:doi
789:doi
748:doi
707:doi
668:doi
632:doi
589:doi
546:doi
510:doi
475:doi
471:274
440:doi
396:doi
359:PMC
351:doi
287:PMC
279:doi
275:324
232:doi
228:107
67:IVR
23:An
984::
894:.
886:.
876:.
866:17
864:.
860:.
805:.
797:.
785:36
783:.
779:.
756:.
744:11
742:.
738:.
715:.
703:12
701:.
697:.
674:.
662:.
638:.
628:11
626:.
603:.
595:.
585:82
583:.
560:.
552:.
542:67
540:.
516:.
506:34
504:.
481:.
469:.
446:.
436:17
434:.
410:.
402:.
392:15
390:.
367:.
357:.
345:.
341:.
320:.
295:.
285:.
273:.
269:.
246:.
238:.
226:.
180:,
172:,
957:.
928:.
902:.
880::
872::
842:.
828:.
813:.
791::
764:.
750::
723:.
709::
682:.
670::
664:3
646:.
634::
611:.
591::
568:.
548::
524:.
512::
489:.
477::
454:.
442::
418:.
398::
375:.
353::
347:2
303:.
281::
254:.
234::
184:/
27:(
20:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.