A Critical Review of Holder

This paper will focus on the longitudinal mixed methods designs which combine quantitative and qualitative methods to look into phenomena that change with time, for example; developmental process, social trends and responses to interventions (Plano Clark et al. 2014). Longitudinal design is best suited to scrutinize the formation of opinion and attitude and social processes (Ruspini, 1999). In mixed methods research, both quantitative and qualitative data is collected, analysed and integrated (Johnson, Onwuegbuzie, & Turner, 2007). Even though the two methods can be used sequentially or concurrently, one can be more or less dominant than the other (Teddlie & Tashakkori, 2009). Van Ness et al. (2011) suggested three models used in longitudinal mixed methods. The prospective model collects data once, qualitatively, after which measures that will be analyzed quantitatively later on are developed. In the retrospective model data is collected quantitatively during different time points and at the last stage the participants’ recollections are collected qualitatively. In a fully longitudinal design quantitative and qualitative data, at all-time points are collected.

Review of sampling framework

Sampling of participants was done with the intention of ensuring diversity of key demographic characteristics like education, income and age. While different time periods are covered by qualitative and quantitative cohort, there is an anticipation that the qualitative sampling framework would be reflective of quantitative data`s wider demographic trends. As such, ratios that are similar of location, gender and age will be used by the sampling framework. Additionally, the sampling framework also takes into account the program’s length of time. The qualitative study, will as stated above sample from a cohort of violence victims aged above 18 years and who participated in the particular procedures of justice despite each having commenced and completed the process at different points in time. Therefore, the length of time will be between 6 and 36 months depending on how fast the cases are solved.

While this sampling framework can be viewed as highly effective and accurate, the researcher did not factor in the differences in terms of numbers at the different time points as a result of attrition.

Sample size

The quantitative study population of the study was 45 individuals who were victims of violence and who started and completed the justice system process at different points in time. With the first interview starting on November 2009 and the last in March 2013, the TIME 1 and Time 3 interviews ranged between 6 and 36 months, depending on how fast each case was concluded. Out of 45 participants who had initially agreed to take part in the research, and 5 not responding to contact with a further 6 indicating availability and 1 more being illegible, the researcher was left with only 33 participants who took part in the Time 1 interviews. Compared to the previous Britain and Netherlands justice studies, this is a relatively small sample size. Due to the limited number of longitudinal mixed research methods on violence victims and the challenges in enlisting participants for research, it is important to share certain findings and challenges arising in methodology.

Methods used to obtain participants

Participants are an important aspect that cannot be overlooked when one is undertaking a research. Sampling is the basic means through which a researcher is able to obtain people to respond to research questions that he or she aims to answer through their study. Research sets out various methods of sampling to get participants to take part in the study, and each sampling method can only be used at certain times depending on the characteristics the researcher would like his or her participants to possess. The researcher could have used either convenience or purposeful sampling or both to identify participants whose cases fit the criterion. Convenience sampling, a non-probability type of sampling, uses the people who are easy to reach and only sets availability and willingness to participate as the key criteria for selecting participants. Purposeful sampling is majorly used for qualitative study and selects participants based on their characteristics/quality and richness of information related to the case of interest ( research purpose) This is due to the fact that random sampling would not be a viable method given the strict parameters and the eligibility criteria specified for this kind of study. However, given the nature of the research and the difficulty of reaching violence victims, the researcher had to rely on local services that were formally involved within the criminal justice process. It was therefore paramount to negotiate a contact and safety protocol with a domestic violence service (Langford, 2000) and likely participants were enlisted only once they had been advised about the research and they had agreed to participate, after which they would be provided with further information. While this method is highly likely to result in small participant numbers, it is the best and only possible alternative that could be used.

Survey and interview schedule design

The interviewer chose to use both closed and open-ended questions during the written and one on one interviews. The interviews were conducted in areas of the participants’ choice, locations that were suitable and accessible for them. The focus of the interview is on the violent situation that the interviewee went through, certain variables relative to the incident and the relationship they had with the offender. Questions focused on the commencement of, reason behind, and experience with police. With quantitative questions being dominant at this stage, Time 1 interviews were rather deductive compared to Time 3 which reflected an inductive inquiry due to the dominance of qualitative questions. The longitudinal design and its fluidity is what brings about this dominance method shift (Strauss & Corbin, 1990; Van Ness et al., 2011). The interview during Time 1 interview was necessary in that it led to the generation of numerous data related to participant demographics, the violence incident, the offending party, and general opinions. To achieve this quickly and with more ease quantitative questions was the best approach, leading to interviews taking between 1 and 11/2 hours to complete. The closed questions also invited individuals to reflect on and talk about realistic meanings and outcomes’’ (Van Ness et al., 2011).

Time 2 interview looked into the different levels in which victims took part with prosecution and with court, and also allowed them a chance to reflect. They were able to clearly think about the judicial process and the justice they wanted, having being well capable of internalizing the violence`s trauma and the police`s reaction. There were more open-ended questions in the Time 3 interview and that required the participants to look back on their experience some period after the completion if the court process. The integration of quantitative and qualitative methods was made possible by the longitudinal design. However, the quantitative dominance observed at Time 1 and the use of repeat measures, needed a larger participant number to be recruited. This was however, not achieved.

In summary, the combined and sequential use of qualitative and quantitative questions in the interview process qualifies the study as a longitudinal mixed methods research. However, further studies could be undertaken and larger samples provided to improve the understanding of different people’s meanings of justice.

Background and participants’ recruitment

The participants recruited for this study were victims of violence who were assaulted and had the violent person charged in the Australian Capital Territory. Choosing to sample only victims of violence served to avoid the obvious differences that came with the many types of violations. The participants were heterogeneous, in terms of income, education, sex and marital status. They were further classified into two groups; female victims assaulted by males or their spouses and male victims assaulted by other non-family males. While a few of the study participants had some previous experience in the justice system as a victim or an offender, majority had little or no knowledge before their present encounter. Because of the difficulty in reaching victims of violence, the recruitment of participants was done through local services formally involved in the judicial process, resulting in the use of safe and ethical contact being a key criterion for recruitment.

Advising participants on the study`s purpose and nature together with the associated potential risks provides them the much needed information that enabled them decide whether or not to participate in the study. Allowing the participants the freedom to withdraw from the study at any time and the ability to choose contact time and interview location also served to give them some power or control over the process. Offering them movie tickets for every interview session can also be said to be a way of incentivizing them to stay on for the entire research period. Recruitment of participants from a single region helped the researcher to minimize any variations that would arise in their institutional experience. It also granted the researcher easy access to the participants as well as the ability to conduct one on one interviews at different times over the long period that the research would last. The move was also seen as a way to tackle panel attrition which is seen as a problem in longitudinal studies (Ruspini, 1999) though Wenzel (2000, p.198) opines that it was not unusual for such studies to have this rate of attrition.

It is worth noting that despite not recruiting a good quantity, the researcher managed to obtain the right quality of participants by way of their having information that would be useful for the study.

Numbers of participants and their retention

This type of research likely needs to have a large number of participants due to the long periods that the research is expected to last. While there was no better way to obtain participants for the study other than what the researcher did, he should have aimed to get more than the 45 respondents who had initially agreed to participate, although only 33 ended up starting at the first time point. There are challenges faced in the retention of participants over long time periods as a result of different factors like the personal circumstances of people like relocations, the closing of schools, schedules that are busy and also the perception of the survey being upsetting are reasons that could participants withdrawing from the study before its finalization. Only 58 percent, representing 19 participants, were retained from Time 1 to Time 3. This panel attrition brings with it the problem of missing data for example, in this case 13 of the victims of domestic assault did not get to Time 3. This means that a lot of data that would have otherwise been generated from this group was not acquired and could therefore not count for much in this study. Majority of those who did not move past Time 1 simply did not respond to attempts to reach them by the researcher for reasons that could not be established but that are likely to fall among the abovementioned. Panel conditioning (a term that refers to the effect on participants that taking part in the study has) is one other issue that often comes up within longitudinal researches (Trivellato, 1999; Van Der Zouwen & Van Tilburg, 2001). The questioning, the interviewer, and the terms used all play a crucial role when it comes to how the participants perceive the study. They could find the study pleasant, upsetting, or unsettling just by the questions are structured or by their perception of the interviewer. It is therefore recommended that the interviewer crates a relaxed environment that would act as a solid ground for the study. By creating a good rapport between them and the participant, a researcher could be able to maintain majority of his original participants and in so doing counter the risk of attrition (Van Ness et al., 2011, p. 294).

Although it is not an easy task maintaining the original participant numbers to the tail end of the research, it has to be admitted that the researcher could have done way better in terms of the initial number of participants since this type of research relies heavily on high participant numbers as a way of dealing with the expected panel attrition during the entire length of the study. Despite the obvious deficiency in terms of participant numbers and by virtue of the fact that the study is largely exploratory, it is recommendable that it has gone on to provide insight into the different perspectives that different individuals hold of justice.

Analysis of data

The analysis of data will serve to examine how the wider contexts of the participants’ involvement in the justice process, their perceptions of the outcome, and their experience with authorities could influence their perceptions and meanings of justice. For this type of study data to be analyzed was both qualitative and quantitative. To analyze the qualitative data, this analysis utilized narrative analysis techniques whose focus is on the ways through which explanations are offered by individuals and the presentation of their personal accounts and as such constructing, identifying and using their stories for the purposes of interpretation of the world. The study also used quantitative analysis strategies which included: those that focused on individual victims (participants) and those focused on groups. To analyze single case quantitative data, the researcher used descriptive methods such as visual analysis by way of graphs and time point changes in scores. For group quantitative data, the researcher employed an array of cross-sectional (such as descriptive) and longitudinal (for example repeated measures analysis and mixed-effects regression models) procedures of statistical inference. He also used tables and figures extensively for results summary.

Validity and reliability

The choice of the longitudinal design is informed by the need to portray a process and a result of its generative potential (Greene, 2007, p. 79). Being longitudinal and mixed design, the research brings out examples of both retrospective and prospective elements at different time points. The responsiveness of the results from this can be said to be a positive attribute of the longitudinal mixed methods research design. Mixed methods have over the time worked to increase attention on certain results, as well as to deepen and contextualize. This being an exploratory study, it was prone to have a tentative generalization. Despite aiming at deeper detail, the small number of participants limited the scope of the study even though it produced results that were consistent with the literature by employing the use of a mix of methods. Further research may be necessary to generate more numbers in order to enable inference to be made casually (Holland & Thomson, 2009).

Problems in using longitudinal research design

As a complex approach used by researchers to conduct research on phenomena that require extended periods of study, the application of a longitudinal mixed methods design that is longitudinal brings about different issues. Such problems arise in areas such as sampling, samples size, data collection, retention of participants and the time factor when carrying out analysis.

Van Ness et al. (2011) raises certain issues that arise when random and purposeful samples are combined due to the implications it would have when it comes to making statistical inferences in longitudinal mixed methods studies. Suárez-Orozco et al. (2010, p. 615) noted that due to the strict parameters set out by the inclusion criteria used by the study, it was not possible to use random sampling.

It is not an easy task for a researcher to maintain the original participant numbers throughout the study. This problem of attrition brings with it the problem of missing data. As a result of missing data and attrition, it becomes a challenge to determine the actual sample sizes used during the study, a challenge further complicated by the use of sample sizes that are different at time points that are different. For example, Holder (2015) started with n = 45 in his quantitative sample at Time 1 but ended up with about n = 19 at Time 3.

A data analysis issues that comes up with this design is time. The time involved in longitudinal research is a critical issue, given that the research collects data at different time points which are likely to have varying sample sizes. The author used a number of strategies in the conceptualization of the time for the study`s longitudinal aspect during the stage of collection of data. Time was treated in two ways: as a continuous variable through the pointing out of time differences between different time points; and also as ordinals variables where time is taken to be a wave number of 1, 2, 3, 4, even in those instances when the time points are not equal. Holder (2015) however does not give an explicit discussion or explanation of how this time is treated during data analysis.

Feasibility and reporting challenges are the other problems that come with this type of design. Longitudinal models were observed by Van Ness et al. (2011) as demanding a lot of researchers’ time and effort. Most authors note their receipt of funding from multiple external sources. With other articles, issues that had relations with the role played by researchers in the collection of data came about. For example, the senior researcher would have to collect one data type and while the assistant collects the other form (Molony et al., 2011).

References

Barbour RS. Checklists for improving rigor in qualitative research: a case of the tail wagging the dog? BMJ 2001;322:1115–17.

Creswell, J., & Plano Clark, V. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.

Denzin, N. (2010). Moments, mixed methods, and paradigm dialogs. Qualitative Inquiry, 16, 419-427. doi: 10.1177/1077800410364608.

Edmondson, R. (2007). Rhetorics of social science: Sociality in writing and inquiry. In W. Outhwaite & S.

Turner (Eds.), The Sage handbook of social science methodology (pp. 479-498). Thousand Oaks, CA: Sage.

Gale NK, Heath G, Cameron E, et al. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013;13:117.

Greene, J. (2007). Mixed methods in social inquiry. San Francisco, CA: Wiley.

Gromet, D., & Darley, J. (2009). Punishment and beyond: Achieving justice through the satisfaction of multiple goals. Law & Society Review, 43, 1-38. doi:10.1111/j.1540-5893.2009.00365.x

Hall, M. (2009). Victims of crime: Policy and practice in criminal justice. Cullompton, England: Willan.

Holder, R. (2013). Just interests: victims, citizens and the potential for justice (Doctoral dissertation). Retrieved from http://hdl.handle.net/1885/11464 Holder, R. (2015). Satisfied? Exploring victims’ justice judgments. In D. Wilson & S. Ross (Eds.), Crime, victims and policy: International contexts, local experiences (pp. 184-213). Basingstoke, England: Palgrave.

Johnson, R., & Onwuegbuzie, A. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33, 14-26. doi:10.3102/0013189X033007014.

Johnson, R., Onwuegbuzie, A., & Turner, L. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1, 112-135. doi:10.1177/1558689806298224.

Plano Clark, V., Anderson, N., Wertz, J., Zhou, Y., Schumacher, K., & Miaskowski, C. (2015). Conceptualizing longitudinal mixed methods designs: A methodological review of health sciences research. Journal of Mixed Methods Research, 9(4), 297-319. doi:10.1177/1558689814543563.

Riessman CK. Analysis of personal narratives. In: Gurbium JF, Holstein JA, eds. Handbook of interviewing. London: Sage, 2001:367–81.

Ruspini, E. (1999). Longitudinal research and the analysis of social change. Quality & Quantity, 33, 219 - 227.

Shapland, J., Willmore, J., & Duff, P. (1985). Victims in the criminal justice system. Aldershot, England: Gower.

Stentz, J. E., Plano Clark, V. L., & Matkin, G. S. (2012). Applying mixed methods to leadership research: A review of current practices. Leadership Quarterly, 23, 1173-1183.

Tashakkori, A., & Teddlie, C. (Eds.). (2010). Sage handbook of mixed methods in social and behavioral research (2nd ed.). Thousand Oaks, CA: Sage.

Teddlie, C., & Tashakkori, A. (2006). A general typology of research designs featuring mixed methods. Research in the Schools, 13, 12-28.

Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research. Thousand Oaks, CA: Sage.

Trivellato, U. (1999). Issues in the design and analysis of panel studies: A cursory review. Quality & Quantity, 33, 339-351.

Van Der Zouwen, J., & Van Tilburg, T. (2001). Reactivity in panel studies and its consequences for testing causal hypotheses. Sociological Methods Research, 30, 35-56. doi:10.1177/0049124101030001003.

Van Ness, P., Fried, T., & Gill, T. (2011). Mixed methods for the interpretation of longitudinal gerontologic data: Insights from philosophical hermeneutics. Journal of Mixed Methods Research, 5(4), 293-308. doi:10.1177/1558689811412973.

Wemmers, J. (1996). Victims in the criminal justice system. Amsterdam, Netherlands: Kugler.

Wenzel, M. (2000). Justice and identity: The significance of inclusion for perceptions of entitlement and the justice motive. Personality and Social Psychology Bulletin, 26, 157-176.

Continue your exploration of YMCA Fairthorne Group with our related content.