Written by Jennifer Leitch, NSRLP Executive Directororiginally published on Slaw, Canada’s online legal magazine.

At a recent conference in Montreal, individuals and organizations involved in access to justice across the country gathered to present current research, share initiatives, and build a strong foundation of knowledge. The conference was grounded in a people-centred approach to access to justice, shaped in part by the OECD Framework and Good Practice Principles for People-Centred Justice.[1] Within this approach there is a common understanding that if we are to improve access to justice we need to include, and in fact, elevate, the voices of those attempting to access justice, both in terms of their challenges, and their experiences. This is neither a radical nor new realization (in fact the NSRLP was founded in 2013 on research into SRLs’ experiences). Over the past decade, many researchers and access to justice activists have called for research that prioritizes the justice system users’ voice. However, despite this call, user-based research is not undertaken as often as it should be.

We continue to ask lawyers and judges and court clerks what they think would improve access to justice or how we ought to operationalize initiatives, which may provide useful insight, but remains, at best, a partial response to the access to justice crisis. Moreover, we struggle to admit that what the legal profession believes will improve access to justice might not be what justice system users need, want, or find practical. Justice system professionals see the problem from a very particular perspective: that of insiders who are trained and experienced in the operation of the law and the legal system as it is currently constituted. Additionally, while the legal profession is tasked with serving the public interest, lawyers are also generating a livelihood. Thus, without appearing too controversial, it is not difficult to see how the profession’s vision of what effective access might look like might, in some instances, be different from the average citizen’s perspective. This is not to say that justice system insiders should be excluded from the conversation about access to justice, or their perspectives deprioritized, but rather that they should not be the only voices heard.

However, beyond the issue of whose perspective is currently sought is a broader concern about how access to justice research is undertaken.

In recent years, we have seen a growth of legal research projects seeking to understand the average citizen’s legal needs, as well as where they turn to secure assistance with their legal problem. Thanks to these types of research projects we now have a greater sense of the nature of people’s legal problems and how they undertake addressing them (or not). To the latter point, this research has also highlighted the fact that many individuals either ‘lump’ their legal problems before even attempting to address them, or give up at some stage prior to resolution.

So, these are the things we know about justice system users. But what do we know about what is working for individuals trying to resolve their legal problems? In our modern age of instant information, we may assume that there are a significant number of individuals who understand that they have a legal issue, and seek to resolve it through a variety of options – we even have a sense of what those options entail. But we do not have a good sense of whether the access to justice resources or programs accessed by such individuals actually assist them with their legal issue. Understanding what works and for whom is crucial to a people-centred approach to access to justice. If we are to develop good resources, initiatives, and pathways, then we need to ask if these materials are indeed helping the people they are meant to serve.

Addressing this question is more complicated than it appears for two reasons: the difficulty of measuring outcomes, and the current perception of what constitutes rigorous and reliable research.

With respect to the first concern, there remains a tension in access to justice research over the measuring of legal outcomes. A variety of factors make it difficult to evaluate outcomes generated by an access to justice initiative. Essentially, the arguments are that there may be too many ‘unknowns’ to isolate the impact of the program, and an ethical concern in controlling for the impact of assistance (in effect the creation of a control group who will not be assisted so that the effectiveness of a program or initiative can be measured). Thus, this type of research is often relegated to customer-satisfaction surveys in which individuals are asked whether they were happy with the outcome. Alternatively, outcome-based research frames achievable outcomes in extremely narrow and sometimes unhelpful ways. The problems these pose are that individuals who feel as if they are drowning will be positive about any life-saving device, no matter how effective, and/or the impact of the program is not reflected in the articulation of the measurable outcome. Thus, self-reported customer satisfaction survey results in this context provide limited meaningful insight into the efficacy of an access to justice initiative.

The second concern relates to the nature of the research undertaken: namely, quantitative versus qualitative research. The tension between these two broad approaches takes on particular weight in the context of access to justice research. Quantitative methodologies are perceived as producing ‘hard data,’ using more rigorous methods, whereas qualitative methodologies invoke more skepticism about the nature of the data generated, what is learned from it, and how it might be deployed. Access to justice researchers continue to debate the rigour of these methodologies (as they should). However, this debate can sometimes result in a form of paralysis, whereby certain types of research are not undertaken, or discounted when they are undertaken. While both methodologies have something to contribute, the reality of large-scale quantitative research is that it may take significant time and resources to design and implement, and as such, is really only accessible to large institutional players who are able to effect access to surveyable populations. Moreover, there are times when it would be helpful if we were focused on understanding individuals’ experiences, rather than measuring the occurrence of a phenomenon; this is particularly the case when outcomes can have both positive and negative elements, and are, as such, complex. In this sense, quantitative research does not allow for a great deal of nuance. This is not to suggest that we do not require quantitative measurements in access to justice research – gathering updated data on the percentage of SRLs in courts is a good example of a much-needed measurement. But a focus on research projects that narrowly conceptualizes ‘rigour’ and does not include qualitative work such as interview and observational research means we are missing out on important insights into the impact of access to justice programming on the very individuals making use of such programming. The result is an absence of data on justice system users’ direct experiences with access to justice initiatives and resources, and lost opportunities to learn and improve on those programs that are working. This is significant when we also remember that many access to justice projects or initiatives are operating on very limited resources, where knowing what works and what doesn’t work becomes urgent.

In order to move access to justice research forward in a truly practical way, researchers, policy makers, and justice system insiders need to embrace the expansion of methodologies that are consistent with a people-centred approach. This is not a discounting of quantitative methodologies, but an acknowledgement that well-designed qualitative research methodologies deployed in the sociological and anthropological context provide a rich canvass of individuals’ experiences and perceptions. This type of research can offer important insight into various aspects of our social world – and these research methods can produce data far beyond that of a simplistic customer satisfaction survey. Where quantitative methodologies can produce data that shapes the scope of the problem, qualitative methodologies with their own internal dynamics allow researchers to better understand individuals’ experiences, and how such experiences impact their lives in both tangible and intangible ways. Moreover, research that explores individuals’ actual experience with a program or process may also identify consequences that are reflective of insiders’ blind spots. While this type of research engages a more complex and nuanced genre of data, it is no less rigorous than its quantitative cousin. Moreover, undertaking qualitative research within established ethical parameters allows us to prioritize the voices of those seeking access to justice, consistent with a people-centred focus.

The challenge for the legal establishment is accepting that rigour, within the context of qualitative, experienced-based research, may look different from its quantitative counterpart. We must be willing to live and grapple with complexity. If we do not, our understanding of the true scope of the access to justice landscape will remain surface level, at best.

______________

[1] See OECD (2021), OECD Framework and Good Practice Principles for People-Centred Justice, OECD Publishing, Paris, https://doi.org/10.1787/cdc3bde7-en.

13 thoughts on “A Little Rigour Can Go a Long Way… Reflecting on Research Methods in Access to Justice

  1. Meg Holden says:

    Thank you for this reflection, Jennifer. As a researcher by profession who became a self-rep out of necessity, I have not only been drowned in court; I have also felt pushed even deeper under water by how closed the legal system is to scrutiny by outsiders like me. Hearing you argue for the validity of voices like mine in reflecting the injustices of the Canadian legal profession is validating.

  2. Ken Grysiuk says:

    35 Injured worker’s across Canada have a charter challenge in from of the Ontario Supreme Court and the sitting judge is completely aware of the wrong doing of the WSIB/WCB.

    The WSIb/wcb legal team wants to have our case thrown out of court despite all the evidence & issues we have been able to point out…

    all because we do not have proper legal representation… it is not fair!

  3. David Price says:

    I am a recently retired judge of the Superior Court of Ontario. I am writing a Guide, for the use of self-represented litigants, to help navigate the family law court system in that Province. I would welcome suggestions as to how such a Guide can meet the needs of srls. Without duplicating existing resources or further overwhelming srls.

    1. Chris Budgell says:

      Dear Mr. Price,
      .
      I’m both a former and a prospective SRL who, though having lived previously in Ontario, has had experience exclusively with the courts in B.C. None of that experience was regarding family law. So I’m assuming that what you intend to produce will offer me very little or nothing.
      .
      I appreciate that there are reasons why so much of the attention to A2J focuses on family law. That may be where the vast majority of SRLs are seen. But the A2J discussion largely ignores the experiences of SRLs pursuing non-family law issues. My perspective tends to be not well received because my experience has been entirely in facing counsel for powerful entities – in fact politically powerful entities, like the B.C. government. And some of us who have been down that road have seen things that no family law litigant sees. I’ll cite one example. This very short slaw.ca column – https://www.slaw.ca/2024/07/30/tips-tuesday-researching-legislation-in-bc/ – with a link to a BC government publication, a printed copy of which I had to pay to receive years ago. This is the kind of information citizens need. It helped me to put together a case in which I alleged and still allege legislation was surreptitiously and illegally amended.
      .
      The A2J discussion needs to be expanded.

  4. Michael Z says:

    I deeply appreciate your efforts to make Canadian legal system more assessable to SRL. I was SRL in BC many years ago and now because of unexpected twist, I was forced to represent myself in ON after I dismissed my lawyer. In my non professional view – ON probably about 20-25 years behind BC on issues of SRL; see below few examples: (a) almost every Supreme Court location in BC has legal library (open to general public including SRL’s) – to the contrary, in ON Superior Courts have NO libraries for the public – ONLY for lawyers (if you want to review /read anything and everything including annual practice) you have to buy it or go to Law faculty library to see it. (b) some time in BC Court library you can find some legal help NO such thing in ON. (c) filing Court documents – BC court registry will provide you with clear instructions…- in ON no simple clear instructions (it took me 5 times to file electronically some documents) Court guidelines say it could be PDF or Word, then they asked resubmit ONLY in PDF. (d) simple application for direction 60 min could be scheduled only in 6-8 month in advance. So SRL must prepare: affidavit, Notice of Motion, serve other party, coordinate time and hearing with other parties and registry and then wait for reply… and then prepare the Factum.
    EVERY SRL faced with very complicated task while he/she has to conduct himself/herself Infront (i) other litigant(s) most time lawyer (ii) Court registry, (iii) filing documents and preparation for hearing including self-analyze legal argument(s) and (iv) actual presentation in front of Judge. SRL’s are not on equal footing with lawyer however if somehow (ii) & (iii) more clear and straight forward, it will help greatly.

  5. Chris Budgell says:

    This article prompted me to take a look first at what the OECD is contributing to the A2J efforts, including a Global Roundtable on Equal Access to Justice scheduled for October at Ottawa’s Fairmont Chateau hotel, but then at the OECD itself – the Organization for Economic Cooperation and Development ( https://en.wikipedia.org/wiki/OECD ) – headquartered in Paris and led by the Secretary-General, since June 2021 Mathias Cormann, an Australian citizen who was born in Belgium and speaks German, French, Dutch and English, see this impressive profile – https://en.wikipedia.org/wiki/Mathias_Cormann. Finding an email address for Canada’s Ambassador to the OECD, Madeleine Chenette, I sent her an email, and then I forwarded a copy to Mr. Cormann.
    .
    In keeping with my usual skepticism I am skeptical that this organization is serious about promoting access to justice. It’s described by others as a “rich-country club”. My guess is that the interest in A2J is something relatively recent.

  6. Karin Turkington says:

    The most difficult clients need help writing their experience to explain to themselves and others why they keep pursuing their case. A sense of injustice is difficult to explain when the rules seem mysterious, despite an abundance of information. What’s happening behind the scenes and in the minds of the players? Who is lying and who isn’t? Not something that can be determined from a website. Writing to understand can take years. I’ve been writing about my experience since 2016 to understand why I live in another province, isolated from my children, and haven’t been employed since I left Ontario. Why am I an alcoholic since 2008 while trying to co-parent with a pathological ex-partner. Determined to give me nothing, he succeeded with the help of his counsel, the mediator, and his father’s influence through the entire process. Having no support system, aside from the Court Support Worker at Guelph-Wellington Women in Crisis, the legal system was all I had. I was at the mercy of those in power who controlled me. I expected normalcy – respectful people using words that were honest and trustworthy. But I received confusing accusations, denials of financial support, violations of an expensive mediated agreement that I had no power to alter without the consent of a controlling ex-partner. The mediator and opposing counsel prevented me from going to court, where I felt I may have received a decision based on law rather than vengeance. Since 2016 I have researched Ontario Family Law, consulted with a former Guelph lawyer whom I respect, as well as a family lawyer in Montreal, where I now live. The legal system expects dishonesty, corruption, and collusion in a day’s work. Practical information gives me little confidence.

  7. Chris Budgell says:

    I’m taking this opportunity today to add another comment about something that has long bothered me. I hope I am not the only person who has noticed that the discussion of the A2J issues – in the media and elsewhere – has died down. More specifically I want to point to one media example. The Globe and Mail is supposedly Canada’s leading English language newspaper. What resources do they commit to reporting on justice issues? Sean Fine is billed as their justice reporter – https://www.theglobeandmail.com/authors/sean-fine/. He succeeded Kirk Makin -https://www.theglobeandmail.com/authors/kirk-makin/, who had that role for many years. If I checked the record from past years I think I’d find what we see this year. Just two articles were published in May. For nearly four months now there’s been nothing. I’m anticipating that we’ll see something from him soon. But he takes the entire summer off. And few of those articles have anything to say about A2J issues. That’s what we get from Canada’ leading English language newspaper.

  8. Harvey Lee says:

    Interesting insights on improving access to justice through rigorous research methods!

  9. Chris Budgell says:

    I have been perceiving for some time a doubling-up of the legal establishment’s efforts to counter what they correctly see as a very serious problem with the public’s loss of faith in the justice system (and not just the justice system – all of our governing institutions). I have just found, published two days ago as an article, an interview the the CBA’s current president, Lynne Vicars – https://www.canadianlawyermag.com/resources/practice-management/the-cbas-new-president-lynne-vicars-on-reinforcing-public-trust-in-canadas-justice-system/388716. I hope others will read the article and listen to the audio. Ms. Vicars has a very inspiring story to tell about herself. That lends credibility to the message. But I had already found this interview-as-article, published on September 3 – https://www.nationalmagazine.ca/en-ca/articles/people/q-a/2024/building-trust-in-justice. So yesterday I sent Ms. Vicars an email. It’s too long to share here, but here’s one paragraph:
    .
    Educating the public is not part of the CBA’s job. You should stop claiming that it is. The CBA was created to serve the interests of its members. No doubt you’ll claim that that is not incompatible with serving the public interest, but I disagree. We still do not have an agency legitimately tasked with educating the public about the law and the legal system. We need one, a competent one that the public can trust. Creating one is something our elected representatives could, and perhaps should, have done a long time ago. On that, and many other issues, they’ve let the public down.

  10. Chris Budgell says:

    Further to my previous comment I want to quote a couple of paragraphs from the September 24 Canadian Lawyer Mag article:
    .
    “First, we will engage our members, particularly young lawyers, empowering them to raise awareness about the role they play in protecting the justice system from misinformation, polarization, and cynicism. Public legal education will be a key focus for the CBA, where we aim to inform the public about the importance of judicial institutions, judicial independence, and the rule of law in a democratic society. We will arm lawyers with the tools and resources to effectively communicate why these principles are essential to everyday life.
    .
    We’ll create opportunities for our members to engage directly with the public so that people not only understand but become champions of justice themselves. This way, they can recognize when these principles are being threatened and understand why they matter. Our goal is to bring justice issues into public consciousness.”

  11. Chris Budgell says:

    Not many people, especially those of us wrestling with the justice system, will be interested in the fact that the Supreme Court of Canada was first constituted in 1875 and that therefore next year is its 150th anniversary. The court is using this opportunity to try to raise public awareness of it. The plans so far include visits to five Canadian cities, starting in February with Victoria – see this announcement – https://decisions.scc-csc.ca/scc-csc/news/en/item/8047/index.do . I think this warrants some attention from anyone concerned about the state of the justice system.
    .
    For a little more insight have a look at the pages starting here – https://www.scc-csc.ca/150/index_eng.html. That includes two short video segments. Note too the reference to Jurivision, with this link – https://jurivision.ca/supreme-court-of-canada/?lang=en. What is Jurivision? See the paragraph at the bottom of this page – https://jurivision.ca/?lang=en. It is an initiative of the University of Ottawa Faculty of Law. The Court is ‘reaching out’ to the public. It is inviting engagement. Hopefully it will be receptive to all honest engagement.

  12. Alice says:

    Canadian judicial council posted on its website the phone number and email address for public to contact them and states ” The mandate of the Canadian Judicial Council is to strengthen public confidence in our justice system. This is why we remain available to respond to your requests.” And”Questions or comments
    Do not hesitate to contact us if you have any questions or comments.

    By e-mail

    info@cjc-ccm.ca

    By telephone

    (613) 288-1566

    I called and left several messages and sent several emails over the several weeks and no one bothered to call me back even once and acknowledge or reply even one of my email. Frustrated enough, I went to the Canadian judicial council physical office several times and the door is always locked and there’s a door bell but no one answering the door when I run the door bell. Why did the Canadian judicial council bother to post the above statements on its website?

Leave a Reply

Your email address will not be published. Required fields are marked *