Reliability and Validity

One issue with regard to enhancing reliability is 'collective analysis', or what Silverman (2014) calls 'inter-rater reliability', defined as "comparing the analysis of the same data by several researchers" (2014, p.90). In relation to this process, this research was designed to reinforce the spirit of collaboration throughout the process. It was based fundamentally on analysing the data collectively with the co-researchers and on continuous interactions with the data set.

Moreover, other strategies I have taken into consideration are those suggested by Morse et al. (2002). They suggest verification strategies for ensuring rigour in qualitative research that is cultivated in the research process specifically, rather than strategies that can be checked by external reviewers at the end of the project. These strategies include verification strategies such as methodological coherence, sampling sufficiency, developing a dynamic relationship between sampling, data collection and analysis, thinking theoretically, and theory development. I have followed all of these strategies in this project.

The first verification strategy is the 'methodological coherence'. This strategy aims to ensure consistency between the research questions, the methods of data collection, and the analytic procedures. Clearly, this is a qualitative project, so the methods of data collection that helped us to answer the research questions were qualitative, such as focus groups and interviews. I found these methods potentially useful in answering the research questions. However, Morse et al. (2002) add that this strategy is not merely about designing appropriate methods: it is also about treating the data differently by changing or modifying methods if required. For example, it was determined in my research proposal that the data-generation method for the last research question (i.e. about the co-researchers' experiences of the project) would be a focus group with the participants who collaborated with me on the project. However, this goal was not achievable. I contacted all co-researchers who collaborated with me, but I was able to meet up with only two of them. Since it was not possible to run a focus group with only two participants, I changed, after a consultation with the two co-researchers, the methods of data generation from a focus group to interviewing the participants as a pair.

The second verification strategy is the 'appropriateness of the sample'. Morse et al. (2002) confirm that the participants should be representative and have knowledge about the topic. As the participants who worked with me on the project were co-researchers, they were familiar with the topic and became more involved in the research process over time. The non-disabled participants and the additional ones who participated in the disability awareness film were also able to gain information about the project and its purpose before taking part.

The third strategy is 'collecting and analysing data concurrently'. For Morse et al. (2002), this process is essential for reaching reliability and validity. The co-researchers and I looked at the data generated directly after each focus group or interview. This interaction with the data was helpful in making decisions about the needs of the process and project.

The fourth strategy is 'thinking theoretically'. This is a concern that the ideas emerging from the data are reconfirmed in the new data (new ideas), which must be verified in the data already generated: "thinking theoretically requires macro-micro perspectives, inching forward without making cognitive leaps, constantly checking and rechecking, and building a solid foundation" (Morse et al., 2002, p.13). One way to achieve this, and thus increase validity in this type of research, is by the use of critical reflection throughout the research process (Melrose, 2001), or what Shenton (2004) calls 'reflective commentary'. Shenton (2004) suggests that reflective commentary includes recording initial impressions of data-generation sessions to help clarify themes generated from the data and theories. As the nature, procedures, and practices of this project suggest, this exercise was crucial in this type of research.

The final strategy is 'theory development'. Morse et al. (2002) confirm that this is about determining whether the data generated in the study can potentially constitute a conceptual or theoretical understanding. Thus, for them, theory can be developed as an outcome of the research process and as a pattern for comparison and further development of the theory. Therefore, determining the continuous interplay between data collection and analysis throughout the research process can lead to the development of a grounded theory (Strauss and Corbin, 1998). This is important, and could lead eventually to a theorising of disability in Kuwait.

Another strategy that I have taken into consideration to verify validation was 'member checking' (Fraenkel and Wallen, 2006) or 'respondent validation' (Silverman, 2014). Fraenkel and Wallen (2006) define 'member checking' as the process of participants reviewing the accuracy of the research report. However, Silverman (2014), by drawing upon the works of theorists such as Bloor (1978; 1983), Abrams (1984), and Fielding and Fielding (1986), reveals that respondent validation of the final report is problematic, as the language used in the final report may be not accessible for research participants or they may not be interested in the topics. Nevertheless, he confirms that this does not imply that the researcher should not seek feedback from the participants. Instead, rather than validating the final report, researchers can deal with feedback on the accuracy of the dialogue throughout the research process as another source of data and suggestions for further analysis.

As this research is based on ongoing interactions with the data throughout the research process, this exercise (i.e. respondent validation) has been taken into consideration. I was able to check the accuracy of the transcripts from the focus groups with the co-researchers. This exercise was useful for us as it helped us to decide on whether we needed to collect more data and the characteristics of the participants who we intended to invite into the project. However, on the other hand, we were not able to check the accuracy of the interviews conducted with the non-disabled participants. I sent the transcripts for each individual in order to check what was said and asked them to provide feedback, but I received no response. This may be due to their busy work schedules, lives, etc., or different priorities.

The Analysis Process

In order to analyse the data, thematic analysis was an appropriate approach that enabled us to answer the research questions. We have followed particularly the six phases suggested by Braun and Clarke (2006) which include becoming familiar with the data, generating initial codes, searching for themes, reviewing themes, defining and naming themes, and producing the report.

Developing Research Skills

The co-researchers stated that one of the motivations behind engaging in this research was to benefit from the research process by developing research skills, such as preparing interview schedules, data analysis, and critical thinking skills.

Kholood confirmed this notion:

These two extracts highlight that Saja and Kholood felt that they had benefited from the research process by developing new skills, such as qualitative research skills and analysis. In fact, developing new skills is important in participatory, emancipatory, and inclusive research; researchers conducting inclusive research who seek to support disabled participants need to use the necessary research skills throughout the research process (see, for example, Abell et al., 2007; O'Brien, McConkey and Garcia-Iriarte, 2014; Stevenson, 2014; Strnadová, Cumming, Knox and Parmenter, 2014). However, relatively few studies have highlighted power relations or supported the developing of research skills (O'Brien et al., 2014). Highlighting power relations and the awareness of where the power lies and who affects it is crucial in this kind of research (Chapman, 2005). In this project, I tried to address power relations by providing the co-researchers the opportunity to acquire some of the skills I have, thus creating a situation where power is more equally balanced (as much as I possibly could in these situations). However, my attempts to provide the co-researchers an opportunity to acquire some of the skills I have could not equalise the balance of power, as in this type of research, one can never equalise the balance of power. In these situations, the participatory research is often led by the academic researcher (Kitchin, 2002).

This study, however, draws upon some of the principles of participatory research by sharing expertise with the co-researchers and supporting them in an effective way (Walmsley, 2001). Furthermore, by highlighting the issues of power relations and skills development, one can explore the complexity that is often ignored (such as failures, frustrations, and difficult times faced by researchers) in collaborative research (McClimens, 2008). The issues of power relations underline the complexities of the researcher's role and raise the question regarding whether the role of the researcher is only to support co-researchers or seek empowerment and make a contribution to developing skills (Chapman, 2005).

In this project, I tried to provide the co-researchers the opportunity to learn the skills required to conduct qualitative research. In Extract 3, Kholood referred to quantitative research by saying that she used questionnaires and that this was the first time she conducted qualitative research. Nevertheless, although Kholood was initially a fan of quantitative research, the next extract (Extract 4) will show how she changed her view over time. In the pair interviews at a later stage, Kholood shared similar views to Saja's:

This extract shows how Kholood changed her view of qualitative research over time by sharing criticism of quantitative research with Saja. They also stated that they had never conducted qualitative research before, talked about the new qualitative skills they learnt (as in Extracts 2 and 3), and specified that Kuwaitis do not care about questionnaires. This may give an indication of the domination of the positivist paradigm and the lack of popularity of the qualitative/interpretative paradigm in Kuwait.

Nevertheless, there is another important reason why Kholood may have changed her mind. Since she knew I was a qualitative researcher, she may have wanted to be included in the project to please me. This was clear in Extract 1 when Kholood stated that one of the reasons why she agreed to participate in this project was to help me with my research. This situation is an example of how the political identity of the academic researcher can have an influence of the co-researchers' preferences (Liddiard, 2013).

Whose Research?

The questions regarding who was in control and whether the members were only research participants or co-researchers occupied my mind a great deal during and after the research process. Therefore, it formed an important theme for discussion in the last two interviews with Kholood and Saja. They both revealed that they believed their roles were important in changing the research path:

The co-researchers believed that they had some control over the research agenda. However, at the same time, in this extract Kholood put the onus on me regarding judging whether something is 'good or not'. This gives impression that she perceived my position as an expert or 'knower' in the field who was more able than them to judge whether certain ideas/suggestions were good or bad. This brings to the fore the dilemma that is often associated with both positivist and interpretative paradigms when the academic researcher is perceived as an expert and 'knower' (Stone and Priestley, 1996). In other words, she perceived me as a "professional as the holder of 'privileged' knowledge and experience against the 'amateur' knowledge of the…[co-researcher]" (Burnett, 2007, p. 6). This extract also raises the question related to whether the ideas raised by the co-researchers truly changed my mind.

My honest answer is no. They did not really change my ideas; however, I would say that the events that happened during the research process (for example, the co-researchers' withdrawal) are one reason why I changed my ideas. It is true that although I had been listening to their ideas and views, I was in control of these ideas. If, for example, their views conflicted with my interest in obtaining my PhD or if they had views that differed from my initial research, this would have raised questions about their potential suitability. This is because of the nature of my research. As I am conducting this research with the University of Manchester, I am restricted by the university regulations and I have to follow my supervisors' guidelines.

Although both Kholood and Saja believed that their opinions had an impact on the research decisions, they were aware of the fact that their role in the research process was related to the disability issues rather than the research methodology:

Both Kholood and Saja believed in the importance of their roles throughout the research process, namely, constructing the research methods and disability topics, yet they thought that I had more control than they did on the research methodology as they believed that I knew more than they did. This may give the impression, again, that the co-researchers see me as an expert in the field, which raises issues of power relations again. This brings to the fore the epistemological assumptions made by academics and the academy in relation to what constitutes knowledge and the way in which 'proper' knowledge is constructed/discovered:

Academics working within the dominant paradigms for disability research (positivist and interpretative) have followed suit, casting themselves in the role of expert and 'knower' – a role which implicitly (and, on occasion, explicitly) maintains that the knowledge and experience of disabled people does not count (Stone and Priestley, 1996, p.701).

I do not think, however, that the issue was solely about knowledge and who knows more about conducting research; it was again an obligation per the university guidelines (that are often based on the positivist paradigm) that I had to follow before meeting them and inviting them to join the project as co-researchers. For example, I was not able to meet them and bring them in as co-researchers before I had prepared my research proposal and passed the panel; this highlights what I would consider a weakness in my attempt to make the project fully 'participatory'. Therefore, I was obliged to prepare my proposal and I decided to go ahead with a qualitative participatory research; I prepared my research questions individually. At the time, I was not able to do it differently (i.e. preparing my proposal collaboratively with the co-researchers) because of time constrains and different locations. This is another form of power relations that is clearly at odds with participatory research principles, and again, it was between the university, my supervisors and me.

A fully participatory research gives "disabled people [opportunity to] take an active role in the whole research process from ideas to hypotheses to data generation to analysis and interpretation to writing the final report" (Kitchin, 2002, p. 3). Including the co-researchers in the whole process was not possible in this research. The idea of making this project participatory was mainly mine. In other words, I decided to make this research participatory and give disabled people the opportunity to be included as co-researchers. This raises a power relation question. When non-disabled academics seek to empower disabled people by giving them an opportunity to participate as co-researchers, are they truly granting disabled people some power and some control or they are actually (because they decided to give disabled people this opportunity) reinforcing their own powers?

Changing Social Attitudes and Societal Structures

The co-researchers discussed the possibility of this research opening the door for further research and changing people's views on disability and disabled people:

In this extract, Kholood seems optimistic about the possibility of this study opening the door for further research and making non-disabled people more aware about disabled people and changing their negative attitudes. Saja confirms the same notion as Kholood, i.e. the importance of raising awareness in society:

Although, Saja expressed the same view as Kholood, her opinions were expressed more strongly. She criticised the country for the negligence of disabled people and the lack of awareness campaigns despite the availability of human and financial resources. Therefore, for Saja, the 'problem' of disability is centred on changing not only social attitudes but also societal structures. She also compared Kuwait with the US, hoping that Kuwait will treat disabled people in the same way as they are treated in the US.

Changing societal structures and policies was one of the reasons why the co-researchers wanted to be involved in this project. They believed that this project might pave the way for social and political change:

Kholood expresses her frustration about policies and creating accessibility. As a result, she sees this research as a potential vehicle to express and share their voices and reach decision-makers and politicians in Kuwait in order to achieve social change in terms of laws and create accessible environments for disabled people.

In relation to the methodology, although a number of inclusive research studies have been conducted around the world (for example, Atkinson, 2005; Chapman, 2005; Abell et al., 2007), this study appears to be the first in the Middle East to provide an opportunity for disabled people to participate throughout the process of conducting the research. In these three extracts, the co-researchers seem to suggest that by giving disabled people a voice to share their experiences, this study can help pave the way for social changes and the emancipation of disabled people in Kuwait. One might argue, however, that seeking the emancipation of disabled people is a rather ambitious goal for this research, yet this project can contribute to this overall goal by providing an inspiration and a basis for further research towards this aim. Thus, this research may not achieve "major change" (Shakespeare, 1996, p.118), but, at least, can start a conversation or 'plant a seed' towards possible social change.

Hearing from the Others

Other benefits of the research centred on identifying the views of both disabled and non-disabled people who participated in this project:

In this extract, Kholood clearly showed the benefits of the study for her by identifying the views of non-disabled people who participated in this research as well as the group of disabled people who have different impairments from hers. Although it was not my intention to explore the issues regarding the nature of the language used by the co-researchers in this paper, it is worth pointing out some aspects of the language that Kholood used when referring to the other disabled people. When referring to those people, she "had created a biography that incorporated … [her] impairment into … [her] sense of identity in such a way as to normalise the impairment" (Watson, 2002, p.517). Although she recognised her impairment, she believed that blind people or those who have difficulties in communication have a more difficult time than she does. By doing this, she created a hierarchy categorisation by placing her impairment in a mild position and placing blindness and communication difficulties in a severe or difficult position.

Although Saja talked about the same idea, namely identifying the views of other people, she focused more on obtaining information about learning disabled people:

Saja had a particular perception about learning disabled people before the study. In this last interview, she said that her view was now different as she used to see them as 'crazy'. In fact, during our meetings and focus groups, Saja would use words such as 'crazy' and 'demented' when referring to learning disabled people, so I am glad that at the end of the study she started to reflect upon her own attitude. However, I do not think that the matter is that simple. It is unrealistic and difficult to claim that one study can change what culture and media have fixed in people's minds over the years. The media in Kuwait shows learning disabled people in a negative way by describing them using inappropriate words and non-representative characters. Moreover, although Saja believed that her views on learning disabled people have changed, the fact that she used the term 'normal' says a great deal about her own 'disablist' assumptions.

Research Difficulties

Research difficulties constitute an important issue throughout the research process, as observed at the first meeting. The issue of the co-researchers' ability to commit remained the main concern throughout the research process. Although I started this project with six participants who agreed to participate as co-researchers, only one co-researcher continued with the project until its completion. Five co-researchers left the project for a variety of reasons, such as personal circumstances related to their studies and families and medical issues requiring treatment abroad (Saja and Abdullah withdrew because they had to travel abroad for treatment). Furthermore, Abdurrahman also decided to leave, but because of personal circumstances. Omar and Yagoub left the group without stating the reason. At the same time, however, the issue of withdrawal may be seen as a natural consequence of the practical realities of life (see the discussion below).

As Kholood was the only co-researcher who kept her commitment to the end, I discussed the issue of research difficulties with her:

Kholood stated that the co-researchers missed many meetings and withdrew as it was not mainly their study and they might have thought they would not benefit from it and waste their time doing it. This issue may be the reason why the co-researchers ultimately withdrew from the study. Therefore, maybe they were committed but just could not make it and then were too embarrassed to tell me. The co-researchers' withdrawal was another expected issue in this type of research, as this study is mainly my own and they were participating as volunteers and were perhaps not interested or might have lost interest during the research process. They were not getting paid and I was getting a PhD (while they got nothing), so why would they be strongly committed? Unlike the projects undertaken within disability organisations, allocating payments for the co-researchers was not possible. However, not all organisations (owing to funding limitation) are committed to allocating payments to co-researchers. For example, in their inclusive project, O'Brien et al. (2014) state that the expenses of the co-researchers were covered but unlike the university researchers, they did not receive wages. However, this was a deliberate act as the researchers stated that payments would have limited the number of co-researchers engaged in the project, even though their future preferences are directed towards allocating payments for co-researchers.

The issue of what I get out of the research and what the co-researchers get out of it is related to the idea of reciprocity. As stated earlier, the co-researchers believed that they gained research knowledge and skills, yet this cannot be compared to what I got out of this research. I gained more benefits than they did, as this study helped me develop my skills as well as obtaining a doctorate and having a better career.

However, I believe that stating the names of the co-researchers would give them credit and recognition in the academic community and this would be beneficial if they wanted to carry on with disability research. Very few inclusive studies have stated the names of the non-academic disabled researchers (see, for example, Abell et al., 2007; Greenstein et al., 2015; Blyth and Docherty, 2015), whereas a large number of inclusive studies have not stated the names of the disabled co-researchers (see, for example, O'Brien, et al., 2014; Strnadová et al. 2014; Stevenson, 2014). For example, in their study which includes four learning disabled researchers and four non-disabled academic researchers, Strnadová et al. (2014) state the names of the non-disabled researchers but fail to mention the names of the disabled researchers. Some would justify this act of negligence considering the ethical procedure that emphasises the confidentiality of information of disabled people. However, if the disabled members were co-researchers (also explicitly referred to as co-researchers in the studies above), then I believe that using pseudonyms is unjustifiable. If the exclusion of the disabled co-researchers names stemmed from a particular educational system and the ethical procedures, then these procedures need to be reconsidered to refrain from blatant exclusion of the co-researchers' identification. However, if the co-researchers are not participating effectively in co-authoring (as in my case) then, at least, their real names need to be mentioned in the data extracts and the acknowledgments section. In this project, I have mentioned the real names of the co-researchers with their consent.

Furthermore, Kholood addressed a vital issue about the type of people who should be involved in this kind of project. She confirmed that I might have come to the wrong place (i.e. the sports club for disabled people) because engaging in sports was the main priority of the members. Throughout the research process, I always tried to ensure that our meeting dates and times were suitable for the co-researchers. In this extract, however, Kholood was trying to say that even the time before/after training was not really suitable, as before training, they needed to be prepared for their training well ahead of time per the instructions of their coach, and after the training sessions, they were tired. Perhaps, this was one of the reasons why they missed the meetings. Kholood was probably right: if I had approached participants who were more interested in the field of Disability Studies, the study would have had fewer drawbacks. This issue was discussed further with Kholood:

Kholood identifies another reason why the co-researchers were less interested: they were working with me as volunteers. She believed that a research study within a particular organisation where an employee's job involves research would be more appropriate and successful.

Kholood stated that it would be preferable to talk to disabled people as they understand their issues better than anyone else. She also stressed, however, that collaborative work between disabled people and non-disabled academics when researching disability is required:

This view is consistent with Barnes' (2007, p.138) suggestion that because "the [disability] movement can provide direct experiences, academics working from a disability studies perspective can provide a logical and consistent political analysis". This is at the heart of participatory and emancipatory research.

Generally, most of the participatory and inclusive projects I came across in the literature were conducted in partnership with disability organisations and even those conducted to obtain doctorates were conducted in universities, but the fieldwork was carried out in a disability organisation (see, for example, Chapman, 2005; Radermacher, 2006). Therefore, confirming Kholood's point, the problems with time restraints, commitment, and interests may be overcome or at least reduced, as within these organisations, it is the job of both the disabled people and non-disabled academics to conduct research to enhance the services for disabled people. Moreover, when the people working together are already well known to each other, the likelihood of achieving positive results is very high. Participatory action research "projects, it seems, are only really suitable in cases where all members are already well known to each other and who know they can work together" (Kitchin, 2002, p.12). In contrast, however, one could argue that there are also definite advantages of not recruiting members from within organisations. On the whole, disabled people in disability organisations are far more aware of disability rights issues, the social model, and activism and therefore will often give far more political views/responses. For example, researchers are required to show their commitment to the core principles of emancipatory research, such as the adoption of the social model of disability (Stone and Priestley, 1996; Walmsley, 2001). Therefore, any deviation by the organisation from the adoption of the social model would be considered inappropriate and erroneous (Shakespeare, 2014). This is often written up as some form of representation of disabled people's views when, in fact, it is a very small minority of very politically aware disabled people. This is one reason why some researchers recruit participants from outside these organisations. Thus, employing disabled people outside disability organisations allows diverse voices and experiences to be heard from people living with their families in homes or in institutions (Strnadová et al., 2014).