Dutch version of the Victorian Institute of Sports Assessment-Achilles questionnaire for Achilles tendinopathy: Reliability, validity and applicability to non-athletes

Abstract

AIM

To translate the Victorian Institute of Sports Assessment-Achilles (VISA-A) questionnaire into the Dutch language (VISA-A-NL), and to assess its reliability, validity, and applicability to non-athletes.

METHODS

After translation according to a forward-backward protocol, 101 patients with complaints of Achilles tendinopathy were asked to fill out the VISA-A-NL at two time points together with visual analogue scale, the Foot and Ankle Outcome Score, and the Short Form-36 questionnaires. Reliability, internal consistency, construct validity, and content validity were tested.

RESULTS

The VISA-A-NL showed high reliability (0.97, 95%CI: 0.95-0.98). Cronbach’s alpha (internal consistency) was 0.80. It increased to 0.88 without activity domain. Correlation with other questionnaires was moderate or poorer.

CONCLUSION

The VISA-A-NL proved to be an excellent evaluation instrument for the Dutch physician. If applied to non-athletes, using a modified score (questions 1-6) should be considered.

Key Words: Achilles tendon; Victorian Institute of Sports Assessment-Achilles; Validity; Patient reported outcome measurement

Core tip: This manuscript shows the validity and reliability of the Dutch version of the Victorian Institute of Sports Assessment-Achilles (VISA-A) in patients with Achilles tendinopathy. The most important finding is that the athletes and non-athletes cannot be compared. The effect of treatment, when using the VISA-A score to measure outcome, is underestimated in non-athletes. If applied to non-athletes, using a modified score (questions 1-6) should be considered.

INTRODUCTION

Achilles tendinopathy is a major cause of chronic pain and disability, which may lead to suboptimal overall health as physical inactivity is a risk factor for cardiovascular disease[1]. Many studies have been published on a multitude of treatments for Achilles tendinopathy, but prospective series on the outcome are lacking. One of the factors limiting the quality of research may be the absence of standardised measures to evaluate the outcome of treatment[2]. A patient’s subjective assessment of treatment outcome such as pain, functional ability, and satisfaction fulfils the criteria of being valid, reliable, and sensitive to change if gathered by a correctly designed and tested patient-centred questionnaire[3]. The Victorian Institute of Sports Assessment-Achilles (VISA-A) questionnaire was created in 2001 to assess clinical severity for patients with Achilles tendinopathy. It is a self- administered questionnaire evaluating symptoms and their effect on physical activity, and displayed reliability and construct validity. Subjective scoring systems can be used in countries other than the ones in which they were developed if translated and validated for a specific language and population[4-6]. The VISA-A has been translated into Swedish, Italian, and German and proved to be able to determine the clinical severity and provide information about the effect of the management of patients with Achilles tendinopathy who speak these languages[1,7,8]. The aim of this study was to translate the VISA-A questionnaire into the Dutch language (VISA-A-NL) and assess its reliability and validity to provide a valid questionnaire for the Dutch population. Moreover, the questionnaire seems to be designed only for athletes as 40% of the points account for activity. As approximately 30% of patients with complaints of Achilles tendinopathy have a sedentary lifestyle[9], applicability to non-athletes was also evaluated in this study.

MATERIALS AND METHODS

Translation procedure

A Dutch translation was made using a forward-backward translation protocol according to the guidelines of Guillemin et al[4,5]. Three people independently translated the English version of the VISA-A questionnaire to Dutch. All three were in the medical field, and spoke English as a second language. One independent native speaker who was not active in the medical field translated this Dutch version back into English. Discrepancies were discussed and adjusted for the final Dutch questionnaire (VISA-A-NL). It was assumed that no major cultural differences in lifestyle exist between the Dutch and Canadian populations, and therefore cultural adaptation of the questionnaire was not required[4,10].

Patients

The local accredited ethics committee (Dutch acronym: METC) reviewed this study in an expedited manner and determined, based on the Dutch Medical Research Involving Human Subjects Act (Dutch acronym: WMO), that the research activities described meet the requirements for exemption from METC review under the WMO. According to the Consensus-Based Standards for the Selection of Health Measurement Instruments criteria, we decided to choose a sample size of at least 100 patients. Actually, 104 consecutive patients from the outpatient clinic of three participating Dutch hospitals were included, of whom 47% were female; 79 (76%) were athletes and 25 (24%) were non-athletes. Their mean age was 48.5 years (SD, 11.6 years).

All patients had complaints of Achilles tendinopathy (including mid-portion and insertional tendinopathy, paratendinopathy, and retrocalcaneal bursitis)[11].

They were clinically assessed and an American Orthopaedic Foot and Ankle Society (AOFAS) questionnaire was taken by the consulting physician. All patients were asked to fill out two sets of questionnaires; the first [A = VISA-A-NL, the Visual Analogue Scale (VAS) pain, VAS function, the Foot and Ankle Outcome Score (FAOS), and the Short Form-36 (SF-36)] was completed on the day of consultation, and the second (B = VISA-A-NL) was completed 5 d later[12]. Additionally, they were asked whether their complaints had changed since the first assessment.

Questionnaires

The original English version of the VISA-A questionnaire as designed by Robinson and co-workers[2] contains eight questions that cover three domains; pain (questions 1-3), function (questions 4-6), and activity (questions 7-8). Scores are summed to yield a total of 100 points in an asymptomatic subject: questions 1-7 score a maximum of 10 points each; question 8, on sporting activity, carries a maximum of 30. Pain on undertaking sports will automatically lead to a loss of 10-20 points.

Since questions 7 and 8 refer to sport activities (accounting for 40% of points) and the study population contained both athletes and non-athletes, we modified the VISA-A score by deleting both questions and investigated the psychometric properties of both the VISA and the modified VISA.

FAOS is a 42-item questionnaire divided into five subscales: pain (9 items), other symptoms (7 items), activities of daily living (17 items), sport and recreation function (5 items), and foot and ankle related quality of life (4 items). Each question can be scored on a 5-point Likert scale (0-4) and each of the five subscale scores is calculated as the sum of the items included. Raw scores are then transformed to 0-100, worst to best score[13].

SF-36 is a self-administered, generic HRQL (health related quality of life) instrument[14-17]. It comprises 36 items across eight dimensions (physical functioning, role limitation due to physical problems, bodily pain, perception of general health, energy and vitality, social functioning, role limitation due to emotional problems, and mental health). The eight dimensions of the SF-36 score are calculated on a 0-100 worst to best scale[18].

The VAS is a 100 mm visual analogue scale and is used to determine the seriousness of pain and functional problems[19].

In 1994, the AOFAS developed a questionnaire to provide a standard method for reporting clinical status of the ankle and foot. The AOFAS-ankle and hindfoot clinical rating system combines both subjective and objective factors into numerical scales to describe function, alignment, and pain. Since objective aspects are incorporated, this questionnaire has to be completed by the investigator[20].

Testing

When a questionnaire is developed or translated, the most important consideration is that it must be able to accurately measure that for which it is designed. To evaluate the psychometric properties of the VISA-A-NL, both reliability and validity were assessed.

Reliability

Reliability is defined as the extent to which patients can be distinguished from each other, despite measurement errors[21].

Test-retest reliability: Test-retest reliability refers to the repeatability of the test and measures the extent to which the same results are obtained on repeated administration when no change in physical functioning has occurred[22]. To determine the test-retest reliability, a second VISA-A-NL (B) questionnaire was given to all patients; 67 patients responded. In 15 patients, complaints had changed at re-test measured by an anchor question (7 item Likert). Test-retest reliability was therefore assessed in 52 patients, using the intra-class coefficient (ICC_agreement, two-way random effects model). An ICC > 0.75 was considered good[23]. A t-test was performed to determine the presence of a systematic difference between the first and second assessment. Additionally, standard error of measurement (SEM) was calculated as the square root of the within-subject variance. The smallest detectable change (SDC) was calculated as 1.96 × √2 × SEM. The SDC is the smallest measurement change that can be interpreted as real change[24].

Internal consistency: Internal consistency of the scale is the extent to which the items are inter-correlated and cover the same construct (homogeneity of the scale). To evaluate the internal consistency of the VISA-A, Cronbach’s alpha was calculated. A Cronbach’s alpha of 0.7 was considered to represent an acceptable degree of internal consistency, 0.8 was considered as good, and 0.9 as excellent internal consistency[25].

Validity

Validity relates to the ability of a questionnaire to measure outcome parameter of interest.

Construct validity: Construct validity was tested by determining the association between the VISA-A-NL questionnaire and the FAOS, SF-36, VAS scores for pain and function, and the AOFAS, using Pearson correlation coefficients. We evaluated construct validity by hypothesizing that correlation coefficients between the VISA-A-NL (with and without the activity questions) and VAS pain, FAOS pain, symptoms, and sport and recreation, and SF-36 bodily pain and physical functioning would be higher than correlations with the other domains.

Content validity: Content validity examines the extent to which all concepts of interest are adequately represented by the items in the questionnaire[12]. It was evaluated by assessing distribution and floor and ceiling effects of the VISA-A-NL. These are considered to be present if more than 15% of responders achieve the lowest or highest possible score[12].

Statistical analysis

Statistical analyses were performed using PASW statistics 18.0 software (SPSS Inc., Chicago, IL, United States). A P-value < 0.05 was considered statistically significant.

RESULTS

Of 104 participants, 11 returned questionnaires that were filled out incompletely or erroneously and were therefore excluded from analysis. Thus, 93 patients were finally included in the analysis.

Reliability

Of 93 patients, 52 (56%) returned the second set of questionnaires. The ICC_agreement of the questionnaire was 0.97 (95%CI: 0.95-0.98) and Cronbach’s alpha was 0.78 for the entire study population (Table 1). A statistically significant difference between the two assessments was not observed for both versions of the VISA-A-NL in athletes, nor in non-athletes (0.29 < P < 0.67).

Table 1 The ICC_agreement and Cronbach’s alpha of the questionnaire.

	Athletes (n = 39)	Non-athletes (n = 13)	Total (n = 52)
ICCVISA total (95%CI)	0.95 (0.91-0.97)	-	0.97 (0.95-0.98)
ICCVISA modified (95%CI)	0.96 (0.92-0.98)	0.98 (0.93-0.99)	0.97 (0.95-0.98)
SEMVISA total	4.41 (4.4%)	-	4.07 (4.1%)
SEMVISA modified	2.64 (4.4%)	2.42 (4.0%)	2.68 (4.5%)
SDCVISA total	12.21 (12.2%)	-	11.28 (11.3%)
SDCVISA modified	7.32 (12.2%)	6.70 (11.2%)	7.44 (12.4%)
Cronbach’s alphaVISA total	0.72	0.82	0.78
Cronbach’s alphaVISA modified	0.83	0.86	0.86

ICC: Intra-class correlation coefficient; SEM: Standard error of measurement; SDC: Smallest detectable change.

Validity

Pearson correlation coefficients of the VISA-A-NL with the other questionnaires are shown in Table 2. The subscale “SF-36 physical functioning” correlated well with the VISA-A-NL questionnaire, both with and without activity domain. Most other physical domains correlated moderately, but the subscales of FAOS quality of life showed a poor correlation with VISA-A. A poor correlation was also observed for the psychological domains (SF-36 social functioning, mental health, role emotional, vitality, and general health perception).

Table 2 Pearson correlation coefficients of the Victorian Institute of Sports Assessment-Achilles-NL with the other questionnaires (with and without activity domain).

	VISA-ATotal	VISA-AModified	VISA-AModified	VISA-ATotal	VISA-AModified
	Athletes (n = 71)	Athletes (n = 71)	Non-athletes (n = 22)	Entire population (n = 93)	Entire population (n = 93)
VAS pain	-0.54a	-0.58a	-0.39	-0.54a	-0.57a
VAS function	0.52a	0.51a	0.44a	0.50a	0.52a
AOFAS	0.48a	0.46a	0.31	0.56a	0.50a
FAOS symptoms	0.45a	0.52a	0.53a	0.58a	0.60a
FAOS pain	0.53a	0.56a	0.52a	0.58a	0.60a
FAOS ADL	0.56a	0.55a	0.47a	0.59a	0.58a
FAOS sport	0.56a	0.61a	0.43a	0.55a	0.59a
FAOS QOL	0.29a	0.33a	0.14	0.38a	0.37a
SF-36 physical functioning	0.55a	0.63a	0.66a	0.70a	0.71a
SF-36 role physical	0.13	0.12	0.58a	0.31a	0.32a
SF-36 bodily pain	0.31	0.40a	0.46a	0.49a	0.51a
SF-36 social functioning	0.01	-0.04	0.29	0.27a	0.21
SF-36 mental health	-0.11	-0.07	0.39	0.21	0.20
SF-36 role emotional	-0.06	0.02	0.65a	0.37a	0.39a
SF-36 vitality	-0.26	-0.25	0.26	-0.05	-0.09
SF-36 general health perception	-0.01	-0.02	0.28	0.25a	0.21
SF-36 physical component scale	0.43a	0.47a	0.36	0.52a	0.51a
SF-36 mental component scale	-0.34a	-0.32a	0.49a	0.04	0.03

^aP < 0.05. VISA-A: Victorian Institute of Sports Assessment-Achilles; VAS: Visual analogue scale; AOFAS: American Orthopaedic Foot and Ankle Society; FAOS: Foot and ankle outcome score; SF-36: Short form-36.

The mean scores of the VISA-A-NL questionnaire were 52.4 (SD 19.7) and 22.0 (SD 15.7) for athletes and non-athletes, respectively. The mean scores of the modified VISA-A-NL questionnaire were 36.2 (SD 13.9) and again 22.0 (SD 15.7) for athletes and non-athletes, respectively. Floor and ceiling effects were not observed in both versions of the questionnaire, as only one (1%) subject scored 0 points, nobody scored 100 points, and one patient scored the maximum of 60 points in the modified VISA-A score.

DISCUSSION

The aim of this study was to translate the original VISA-A questionnaire on the subjective complaints of patients with Achilles tendinopathy into the Dutch language, to validate it, and to assess its applicability to non-athletes.

The translation procedure did not create any problems, since the items are universal and there is no large cultural difference between Dutch and Canadian patients.

Reliability of the translation was excellent, with a statistically non-significant difference between assessments. This outcome may be explained by the fact that, as the procedure for the Dutch Oxford 12-item knee questionnaire taught us, we introduced a question if complaints had changed between assessments[26]. Fifteen (22%) of sixty-seven patients answered this question with “yes”, and therefore they were excluded from reliability testing.

The SDC in this study indicated that under a stable condition, the VISA-A score can vary up to 12 points. For clinical studies, this implies that clinical changes can only be detected if they exceed the 12 points.

The study questionnaire showed good internal consistency (Cronbach’s alpha = 0.80), but there indeed was a negative effect of the activity domain. Sub-analysis without these questions showed an increase of Cronbach’s alpha to 0.88. However, when measuring the effect of a treatment in a mixed group of athletes and non-athletes, the effect of treatment could be underestimated. For example, an athlete can score 0 points with questions 7 and 8 before treatment as complaints withhold him/her from being sports active. After treatment, the athlete is complaint-free and scores 100 points. The non-athlete also scores 0 points for questions 7 and 8 before treatment, and is also complaint-free after treatment. However, he/she will never score higher than 60 points as questions 7 and 8 will not be answered differently between assessments. The effect of treatment, when using the VISA-A score to measure outcome, is therefore underestimated in non-athletes.

When choosing the VISA-A questionnaire for a mixed population of athletes and non-athletes, deleting questions 7 and 8 can be considered, since the psychometric properties of the modified VISA-A were comparable with those of the original version.

Generally, Pearson correlation coefficients were higher for physical than psychological components. Convergent and divergent validity is confirmed as correlation coefficients were higher for the physical domains of the questionnaires. However, for non-athletes, correlation with socio-emotional components was also higher. This could imply that physical restrictions in this subgroup with chronic Achilles tendinopathy have greater emotional and social consequences than in athletes. Noticeable is the moderate correlation of VISA-A with VAS, which is a validated subjective outcome measure frequently used for scientific means.

Low correlations can be explained by the fact that none of the questionnaires except for the VISA-A were validated for Achilles tendinopathy. Initially it was intended to do so, but this study aim was departed to not further enlarge patient burden as many of these questionnaires are extensive. Given the laborious inclusion of 104 patients in 3.5 years and a 56% response rate to both assessments, this was well decided. This was also why responsiveness was not tested.

In conclusion, the VISA-A-NL questionnaire seems suitable for use in athletes. However, in a combined population of athletes and non-athletes, results will become incomparable as the highest possible score for non-athletes is 40 points lower than that for non-athletes. Psychometric properties of the VISA-A-NL, without questions 7 and 8, are satisfactory for both athletes and non-athletes. It is therefore proposed that the modified VISA-A-NL questionnaire is considered a mixed population of patients with Achilles tendinopathy, meaning a version with only questions 1-6 and the complete questionnaire (questions 1-8) is reserved for athletes only.

ARTICLE HIGHLIGHTS

Background

Achilles tendinopathy is a major cause of chronic pain and disability. Many studies have been published on a multitude of treatments for Achilles tendinopathy, but prospective series on the outcome are lacking. The Victorian Institute of Sports Assessment-Achilles (VISA-A) questionnaire was created in 2001 to assess clinical severity for patients with Achilles tendinopathy. It is a self-administered questionnaire evaluating symptoms and their effect on physical activity, and displayed reliability and construct validity.

Research frontiers

The VISA-A has been translated into Swedish, Italian, and German and proved to be able to determine the clinical severity and provide information about the effect of the management of patients with Achilles tendinopathy who speak these languages.

Innovations and breakthroughs

This work translated the VISA-A questionnaire into the Dutch language (VISA-A-NL), assessed its validity and reliability, and provided a valid questionnaire for the Dutch population. The questionnaire seems to be designed only for athletes as 40% of the points account for activity. As approximately 30% of patients with complaints of Achilles tendinopathy have a sedentary lifestyle, applicability to non-athletes was also evaluated.

Applications

Psychometric properties of the VISA-A-NL, without questions 7 and 8, are satisfactory for both athletes and non-athletes. It is therefore proposed that the modified VISA-A-NL questionnaire is considered for a mixed population of patients with Achilles tendinopathy.