VIRGINIA BAR EXAM: LACK OF TRANSPARENCY

Update 36.  Last month I provided the deans of Virginia's law schools with the twenty-page expert report in relation to my case and all candidates who experience similar technical problems saving their bar exam essays using the Exam4 software.  Also included were my sworn statement as to what I experienced at the test site (and observations of others) and print-offs of the policies of neighboring state board of bar examiners who, unlike Virginia, recognize the due process rights of the applicants and allow access to the graded essays following the exam.  See, for example, Maryland's Rule 19-207, available at http://www.courts.state.md.us/ble/pdfs/baradmissionrules.pdf

Last year, when the Courts Committee held a hearing on this, the Board told them that it has decided to change its unwritten policy of not allowing anyone access to their essays to allowing a review of the essays after second and subsequent failures, but still not after the first exam.  This is unacceptable and poses no remedy to applicants like Eric Zeni in New York, who was told his essays were properly received only to discover that was in fact not the case once he actually obtained his essays as of right. Mr. Zeni also provided a well written letter to the Courts Committee in Richmond explaining how critical the right to obtain his essays in New York was to identifying and correcting a grading mistake as a result of a technical failure of the board's software (his symptoms were very similar to what we are experiencing in Virginia). 

With the help of friends, an analyst for a federal intelligence agency and a lawyer for the Federal Reserve, I have prepared the following letter presenting the issues to the deans of the law schools of Virginia.  It is time for Virginia to join the rest of the States in ensuring transparency in the system and guaranteeing the due process rights of the applicants.


December 3, 2016

Re:  Transparency of the Virginia Bar Exam

1.  Virginia does not allow an unsuccessful candidate to obtain their essay answers or review them until a second failure, even in situations where there is evidence of a system software malfunction or other technical failure in the grading.  This undercuts the spirit and purpose behind Virginia Code Section 54.1-3929, which requires the Board to preserve the essay answers for one year.

2.  This is fundamentally unfair, makes little sense, and is contrary to the recommendations of professional associations, the approach taken by 43 other states, and the analysis of outside experts.

3.  This policy seriously adversely impacted myself and others, and will continue to do so.  It reflects poorly on the Commonwealth, calls into question unnecessarily the credibility of the exam results, and can create the impression that Virginia law schools are lagging in successfully preparing their students to pass the bar exam.

4.  It is high time for a change in policy now that the bar exam has made the transition to being administered as a computer-based test.


Dear [Dean]:

As Justice Brandeis once said, "Sunlight is the best disinfectant."  I am writing to you and the other Virginia law schools in reference to a matter that directly affects the integrity of the bar admissions process in Virginia, a problem that directly impacts law school rankings and has gone largely ignored by the courts and the legislature after it was presented to them.  And that is the existence of an unwritten rule that no applicant can access their bar exam essays after their first bar exam.  This rule was applied to me after I experienced a system software malfunction and reboot on Exam4 software provided by the Board of Bar Examiners (see my sworn affidavit enclosed and, separately, article published in the Computer Law & Security Review by myself and a leading computer forensics expert).  As you will see when you read the article, these software problems are occurring nationwide and are directly tied to misgraded bar exams (fn.  for example, see this story of a New York bar applicant who experienced a software problem and how he was able to protect his interests by obtaining his essays as of right:  http://www.newyorkpersonalinjuryattorneyblog.com/2008/02/i-passed-the-new-york-bar-exam.html).  I have no doubt that my essays were impacted, as the essay score was nowhere near how I had actually performed.  Shortly after the exam I was able to reconstruct my essays by memory and was able to compare them to the approved solution.  The relatively recent national transition to computer-based testing is therefore a significant reason for a policy change at this time for the Board to allow applicants to obtain their essays upon request, especially if something went wrong at the test site (fn.  this would be an inexpensive proposition, as the other states are simply charging a nominal fee to cover the costs of making the copies.  Florida, for example, charges $50; North Carolina $20; New York $40).   43 of the states have made this transition already (see enclosed chart) given the inherent risks of computer-based testing, and Virginia would do well to follow suit in order to protect the due process rights of the applicant.

I brought a constitutional challenge to the policy of withholding the essays from review in federal district court in Richmond and filed a 20-page expert report in support (see report enclosed).  This report was filed pro bono by Dr. Stephen Castell, a leading computer forensics expert with over forty years of experience as an IT consultant in some of the highest profile cases in the world.  For example, in 2012 he led a team of three experts in  Australia's T-Card highway transportation system failure (ITSL & ERG v. PTTC, a $200M claim against the government for contract termination was ultimately settled on the eve of trial largely due to Dr. Castell's involvement and findings).  See http://www.smh.com.au/it-pro/government-it/tcard-battle-settled-out-of-court-20120217-1tegm.html.  He also testifies in high dollar patent cases in American federal courts for law firms such as Jones Day and Morrison & Foerster.  Virginia is certainly very fortunate to have him looking at this issue (cv attached).  Dr. Castell describes how my case was a "textbook case" of a system software failure and how there is no way to properly determine completeness checking without including the applicant in the process.  In Par. 20 of the expert report he states as follows:  "In no way does this pose a remedy to, or even a sensible investigation of, any potential corruption problem caused by the Exam4 software itself . . . for which Mr. Bolls could not have been responsible."  I have consulted two additional experts who have arrived at the same conclusion as Dr. Castell, which may be one reason why the Board has never brought forward an expert witness of its own.  In the end, the case was dismissed for lack of subject matter jurisdiction, but one good aspect that came out of it was this:  in their Answer, the Board admits to having made a uniform and unwritten rule that no applicants may have access to their essays and that rule was applied to the circumstances of my case.  I have enclosed a copy of my sworn affidavit on what took place at the test site for your review as well, which clearly affected many others as well as myself.  To this day, my essays remain cloaked in secrecy, encrypted on my laptop, and I have never seen them to verify completeness (fn.  it is also interesting to note that Virginia Code Section 54.1-3929 requires the Board to preserve an applicant's essays for one year.  The Board, however, has interpreted discretion accorded to it in another code section as license to make a uniform rule that no applicant can access their essays.  This, of course, is not a proper use of discretion). 

Meanwhile I took my MBE (multiple choice) score, which was high enough to waive into other jurisdictions, and waived that portion of the test into Washington, D.C.,  taking the remainder of the essay exam only.  I successfully passed in D.C. on my first attempt, in a test where over half of the applicants failed.  I then took the Maryland Bar Exam and passed on the first attempt as well, scoring highly.  I have been working as a full-time magistrate in Fairfax, VA.  I also served as president of the Washington Inn of Court in D.C. last year and I teach trial advocacy at a local prep school.  So fortunately things here worked out for me despite the actions of the Virginia Board of Bar Examiners (fn.  a member of the admissions committee of the Maryland Board of Law Examiners actually apologized to me for what the Virginia Board did and said they hope that I will have a better experience with the legal community in Maryland).  Nevertheless, I am cognizant of the fact that the facts of my case militate so strongly in favor of a change in policy at this time to allow applicants access to their essays in a computer-based test (which are preserved for a year by Va. Code Section 54.1-3929) that it would be unfair to future applicants for me to look the other way on this issue.  As previously stated, I was one of many applicants who experienced the software system malfunction and presumably all of them were treated the same way I was if they had later asked to see or obtain their essay answers.

To borrow the words of the computer forensics expert:

"Virginia, as I understand it, does not allow applicants to obtain their essays.  In my view and experience, this is an unusual way to design a system and immediately gives rise to concerns over validity and completeness checking, and difficulties in identifying and assessing any data corruptions or omissions potentially caused by system malfunction or otherwise.  Clearly, to apply such a policy, even to the circumstances of Mr. Bolls' case, is to overlook the possibilities of systemic error and to have no objective and evenhanded methodology of investigation in regard to ruling out such possibilities.  Mr. Bolls knows what he wrote and, with his input, it may be discovered that software malfunction was the cause of the discrepancy.  Without an allowance for applicants such as Mr. Bolls to obtain their essays, in my opinion the board may very well have a serious problem on its hands and not even know it."   Par. 30 of expert report.

The Board saw all of this information, including my sworn statement as well as the expert report; nevertheless, they still refused to release the test essays, even though they had the discretion to do so.  When asked by senior Circuit Court Judge Alden in Fairfax why not, they told the judge in open court that they used their discretion to make a blanket rule that no applicant can access their essays.  This is why when it was ultimately determined that the Board was exempt from the Administrative Process Act, in her dismissal the judge made it clear in her written opinion that she is preserving the issue of whether the Board abused its discretion in making a blanket rule that no one can review their essays.  As it stands, I was not able to view my essays following the system reboot at the test site and I have not been allowed to see them ever since.  Furthermore, due to my blog which tells this story (and has nearly 11,000 hits from the legal community), I have been informed by applicants as recent as last year that these software problem persist at the test site for many people.   Presumably, all of them are being offered the same sham remedy that I was (see below, an offer I did not accept as this would lead to a misleading result).  The Board's lack of cooperation is likely a result of not wanting to employ the National Conference of Bar Examiners' alternative grading methodology for when a software malfunction occurs.  See Par. 5 of http://web.archive.org/web/20071123015445/www.nybarexam.org/press.htm.  This would necessarily include all applicants affected, which they consider an inconvenience to their usual way of doing things prior to the advent of computer-based testing.

SHAM REMEDY OFFERED BY THE BOARD

The sham remedy being offered to applicants who experience a software system crash while the essays are being saved is as follows:  compare the encrypted file on their Exam4 software (saved on the applicant's computer) to what was graded.  But herein lies the problem:  the file saved on the Exam4 program on the applicant's computer itself is likely corrupted to begin with.  Therefore, what the graders graded was necessarily a duplicate of such potentially corrupted data.

As Dr. Castell states in his report:

"It is my opinion that the Board's offer to compare Mr. Bolls' encrypted file against the one graded does not in any way address the matter at hand.  That is to say, an exact match between the two does not inform the board or the Supreme Court of Virginia that there was no data corruption in this instance.  It merely confirms that the work is that of Mr. Bolls, without providing any illumination as to whether or not the work itself had been corrupted.  The only way that I can see this can be reasonably resolved is for there to be a policy in place for applicants to obtain their essays."
Par. 36 of the Expert Report

"Like Mr. Zeni, applicants such as Mr. Bolls would and in my view should then have the opportunity to identify places where portions of the essays went missing or were corrupted in some way.  If, as in my view, there were in Virginia to be the reasonable, and technically sound, policy in place for applicants to obtain and inspect their essays, then, if such portions were thereby identified, it seems to me that there would then be an obvious and sensible remedy, viz to employ the alternative grading methodology developed by the National Conference of Bar Examiners." 
Par. 38 of the Expert Report

What ends up happening is most applicants, generally unfamiliar with computer data storage methods, will take the Board up on their offer to do this comparison exercise, and be naively satisfied with it.  In reality, had they actually seen their essays, it is likely they would be seeing what the applicants in New York observed in 2007- missing or altered data that impacted the grading.  This is what Mr. Zeni experienced after facing a similar reboot of his software in the middle of the bar exam in New York.  See the enclosed letter from Mr. Zeni to the Courts Committee in Richmond.

Because of my experiences with the Board, I no longer have confidence in the integrity of bar admissions in Virginia, so rather than retaking the exam I have decided to go to other jurisdictions to become barred.  However, I have recorded every stage of my legal case as well as a hearing before the Courts Committee on my blog.  Included in the packet for your review are the proposal and other materials set before the committee.  Ultimately, the committee concluded that my case, while badly handled by the Board, was not enough to take action that would ensure applicants rights to review the essays.  However, it is important to remember that I was certainly not the only one affected (and many more continue to be affected), and our test results remain unsolved mysteries in that first exam sitting.

Since the litigation of this case, the Board has changed its policy to allow applicants to review their essays on the second or subsequent failure; however, the first bar exam is the most important.  This is the exam where many applicants (like myself) have jobs on the line; law school rankings are also impacted by the first exam results.  It is essential that the first bar exam be graded accurately and any software failures include a satisfactory review which according to my expert (and all other experts that I have consulted) necessarily includes the applicant in that process.  It is no longer proper for a computer-based test to have a rule that no applicant can review their essays afterwards, and it is high time for a change in policy.

The American Bar Association, the National Conference of Bar Examiners, and the American Association of American Law Schools have drafted a "Comprehensive Guide to Bar Admission Requirements 2016."  See http://ncbex.org/publications/bar-admissions-guide (page X).  In Section 25 it has this to say:

"25.  Rights of Failing Applicants . . . [a]n applicant who fails a bar examination should have the right, within a reasonable period of time after the announcement of the results of the examination, to see the applicant's answers to the essay questions and the grades assigned thereto, and to compare each of these answers with an approved answer." 

I have included in this packet the printed rules of surrounding jurisdictions Maryland, District of Columbia, North Carolina, and New York that adhere to this best practice.  These are but a small sample of the vast majority of the States that have the same transparency protections in place for the applicants (see chart, enclosed). 

If you are interested in this subject and in using your influence to help Virginia join the 43 other states who have already protected the due process rights of the applicants, I would be happy to discuss or answer any questions you have of me.  What needs to be done to fix this problem is what most of the other states area already doing- to allow applicants to obtain, not just review, their essays.  And this must be after every examination, not just second or subsequent failures (see legislative proposal, enclosed).  We live in a time where there is such a high degree of reliance on computer systems, which has many benefits, but we must ensure that our policies are appropriately tailored to the changed circumstances to ensure that rights are protected while the Board of Bar Examiners performs its functions.  Indeed, that was the clear intent of the General Assembly when they enacted Va. Code Section 54.1-3929, which requires the Board to preserve essay answers for one year.

Thank you for your attention to this matter and I look forward to hearing from you.


Regards,

Jonathan Bolls
Magistrate, 19th Judicial District
Fairfax, VA

Cc:  the law schools of Virginia

SEE LETTER BELOW

Update 35.  On February 1, 2016 this matter was brought to the attention of the Civil Courts Subcommittee by Delegate Vivian Watts of northern Virginia.  After looking at the facts of my case, she agreed that the time has come to bring transparency to the Virginia Board of Bar Examiners, especially now that it is operating a computer-based test.  In Richmond, she brought forward House Bill 550, which is to ensure that all applicants have access to their essays as originally intended by the Legislature.  I was also called as a witness to the system software failure that took place affecting me and many dozens of others at the test site who were then told we could not see what impact there was on our essay answers.  Present at the hearing were the committee members, led by Chairman Greg Habeeb of Southwest Virginia as well as an audience of Virginia residents.  Delegate Habeeb is very public about the need for transparency and accountability in government.

Delegate Watts spoke first, explaining that the bill is in line with what 43 of the States are already doing and offers much needed transparency and accountability in the grading process.

I explained to the committee that I serve as a magistrate and have since passed the bar in multiple jurisdictions, on the first attempt, and scored highly.  I am and have always been certain that the essay score I received for the Virginia Bar Exam was inaccurate and highly likely a result of the system software malfunction I and numerous others experienced at the test site.

In my statement I first laid out a blow-by-blow account of what happened at the bar exam with the Board’s software- how it froze during the saving stage of the essays and how a technician went to work hands-on but was unable to fix the problem and finally, as a last resort, instructed me to reboot the system (turn it off and on again) and then save to a USB thumb drive without looking at the essays.  One of the delegates blurted out, “You should have filed for an injunction!”  I later went into my legal case and so he was able to find out that is exactly what I did do. 

I then provided each member a chart of all the States that do allow access as well as a personal letter [see letter below] addressed to the committee and written by an applicant in another jurisdiction who experienced the same thing we did here in Virginia.  The difference was that he was able to obtain his essays by right.  When he got them, he discovered portions were missing, and indeed, one answer response was completely erased.  His answers were graded as if those were his complete answers.  Since 48 others who obtained their essays after his test had similar observations of erased material, this forced a corrective process in the state of New York not previously in existence.  Importantly, prior to the applicants obtaining their essays they were reassured by the Board that they need not be concerned and that their complete answers were received.  No so.

The delegates were additionally provided with a 20-page expert report submitted in support of my case.  Dr. Stephen Castell, a leading computer forensics expert who has testified in high dollar cases for companies such as Motorola, Citibank, and Harvard Securities PLC, on both the Plaintiff and Defense side, was able to form a conclusion that, given what happened at the test site (and still does in the most recent exams from what I am told by multiple sources), my concerns were well justified.  Further, he says there is no way for the Board to properly investigate without including the applicants, the actual writers of the essays, in the review.  As he says, “they [the Board] may have a very serious problem on their hands and not even know it.”  In other words, all of the tests of those affected remain unsolved mysteries. 

Just last year ExamSoft Worldwide, Inc., the sister software company to the one that Virginia uses (Extegrity) was sued in a class action by the applicants for software failures at a test site in another jurisdiction, resulting in a multi-million dollar settlement.  See http://lawschoolcafe.org/2015/05/20/examsoft-settlement.  Case No. 1:14-cv-22950.  Protecting the software company may be one reason for the lack of cooperation with Virginia bar applicants who are falling prey to system software malfunctions and who notice a significant scoring discrepancy in how they believed they performed on the test.  

Finally, I shared some research with the committee I conducted on the history of Va. Code Section 54.1-3929, which since 1919 has required the Board to preserve the essays for one year.  Sure enough, I discovered that prior to 1973 applicants in Virginia were, in fact, able to have access to their essays, which was the fundamental reasoning behind requiring the Board to preserve them for a year.  Sometime after 1978 the language pertaining to the right of access was removed, leaving an absurd result of the Board being still required to preserve the essays but the applicants left unable to access them, even in the most compelling circumstances.  This paradox lays the groundwork for the ongoing violation of Due Process in the Commonwealth of Virginia for anyone who has a legitimate dispute with the Board.  The very evidence that is needed, i.e. the essays, is being withheld in order to prevent a fair hearing.

I left them with this: even if the right to access the essays was for the sole purpose of allowing applicants to be able to prepare and improve the next time, is that not good enough reason alone to get rid of the blanket rule that no applicant can access their essays?  

 

The committee seemed a bit lackadaisical unfortunately and not as focused as one would like on the important issues at hand.  Chairman Habeeb seemed to actually be the only proactive one.  He first said, as the matter was being called forward, that this ultimately has to do with whether we trust the Virginia Board of Bar Examiners.  Some did take an interest when I explained the procedural history of my legal case and how venue was changed from the Fairfax Circuit Court to the Virginia Supreme Court due to the fact that the Board managed to have itself exempted from the Administrative Process Act (which reviews for agency wrongdoing).  And the Virginia Supreme Court has no original jurisdiction and dismisses all of these cases as a matter of course.  This is why, as I explained, the Board behaves this way towards people as opposed to other professional licensing boards, such as the Architects, which deal directly with public safety.  There is no doubt that if a candidate for an architect license experienced a software glitch in the computer-based test, and their board refused to cooperate with that applicant, a hearing could be heard in Fairfax Circuit Court.  Should it be any surprise then that the Board of Bar Examiners, as the only such organization shielded from judicial review, would treat people so poorly? 

 

The new Secretary of the Board, Catherine Hill, spoke against the bill.  She cited to a lot of statistics about the pass rates, etc. which really were beside the point for the most part.  She decided not to address any of the computer forensics evidence and simply argued for the status quo, which she argued was working just fine.  When asked by Chairman Habeeb if she has looked into this problem, she said yes and said vaguely that there was no problem found.  There should have been a follow-up question as to how did you look into it and what technical expertise was brought to bear in any supposed review of the situation.  Instead, in response to the actual follow-up question of whether there have been any statistical anomalies over the years, she simply said no. 

Besides these softball questions, there unfortunately wasn’t much in the way of critical analysis.  Any semblance of rigorous debate to get at the truth of this was wholly lacking.  Either the committee was ill equipped to handle a matter of this complexity or there was a simple deference to a government agency (many of them are licensed by the Board). 

Whatever the case may be, the committee tabled the issue from any further consideration this year.  At the end, Chairman Habeeb told the Board (which showed up as a team) they “really do not want us legislating this so make sure that people are happy.”  Folks in the audience could be heard in agreement with that statement.

 

Judging by one of his comments, Delegate Habeeb revealed some of his rationale by saying that we do not want to make bad law based on a badly mishandled case.  While that may be true in certain instances, that is certainly not the case here for two reasons.  First, I was by far not the only one affected by the system software failure; and second, these problems are persisting up to the most recent exams I am told by some who follow this blog.  The truth of the matter is that badly mishandled cases will continue unless a good law is made now.  It has worked just fine to require transparency in almost all of the other States. 

 

But, as we joked at the hearing, we all know that Virginia is usually the last to change.  Here was Virginia’s opportunity to do something right for the People, but passed up the opportunity.      

 

 

LETTER FROM ERIC ZENI, ESQ TO VIRGINIA GENERAL ASSEMBLY

January 30, 2016

Courts Committee
General Assembly Building
201 N. 9^th Street
Richmond, VA  23219

 

Re:  Virginia House Bill 550

 

Dear Members of the Courts Committee:

 

I would like to submit this statement in support of House Bill 550, which would allow bar exam applicants access to their essays.  I am an attorney admitted to practice in New York State.  I sat for the July 2007 administration of the New York Bar Exam, which I registered to take using a computer.  I believe that my experience with the New York Bar Exam shows why it is important to make the applicant’s answers available to them.

 

When I sat for the bar exam, there were wide-spread problems with the exam software that was used.  In my testing location, a great many people were affected by the software glitches.  Later we learned that these problems occurred state-wide.

 

The exam software froze twice while I was taking the exam.  As I was typing my first essay, the software froze and a technician came to assist.  The technician restarted my laptop, which I did not think was a good idea, but eventually got the software running again.  I then completed the first essay and moved on.  As I was half way through the second essay, the software crashed again.  A technician came and advised me that he could take my laptop to a table in the back of the exam room and work on it.  I glanced back to see a table in the rear corner of the room with computer technicians, equipment, and anxious test-takers- a veritable computer triage area.

 

At that point, I told the technician to give me an answer booklet.  I completed the remainder of the second essay and finished the exam on paper, the old-fashioned way.  At the end of the day, we were instructed to transfer our files on a thumb drive and upload them.  At the conclusion of the test, I spoke with the technicians and the head proctor who advised me that the software had a “backup system” and that no data would be lost.  I believe that I even received an email confirmation from the software company that they had received my test responses.

 

Ultimately, when the exam results were released, I was told I failed by four points.  At that time, in New York, test-takers who failed were allowed to request copies of their answers.  After I received the copies of my essays, I discovered that my response to the first essay written on the computer was incomplete and that the first half of my response to the second essay (which I had typed on my computer) was completely missing.

 

Although New York does not allow appeals of the test results, I contacted the Board of Law Examiners and explained that significant portions of my responses were missing and explained the software malfunctions that had occurred.  Several weeks later, the Board of Law Examiners informed me that they had done some sort of “review” and were able to locate the missing portions of my essays.  My essays were re-graded and I was informed that I did, in fact, pass the bar exam.

 

It is important to note that if New York did not allow an applicant who failed to request and review their essays, I would never have discovered (or been able to prove) that the graders did not have my complete answers and that the software malfunction had resulted in the loss of significant portions of my essay responses.  The ability to review my answers was truly life-changing for me- it meant that I did not have to sit for the bar exam again and I was able to be sworn in on admission day with my peers and begin my career.

 

This is why I write in support of H.B. 550, which I believe will protect the interests of the applicants.  I believe that it is only fair that applicants have access to their responses, especially if they need to prepare to retake the examination.

 

If I can be of assistance to your committee, please feel free to contact me if you have any questions.

 

Sincerely,

S/

Eric Anthony Zeni

VIRGINIA BAR EXAM: LACK OF TRANSPARENCY

Update 34.  My letter to the Virginia General Assembly has been received favorably and a timeline has been requested of me as well as the name of the chief justice of the Va. Supreme Court at the time the case was presented, Leroy Hassell.  Below is the procedural history of the legal case I brought on behalf of myself as well as all applicants for the Virginia Bar:

DECEMBER 2008    FAIRFAX CIRCUIT COURT.   Between the end of October and beginning of November I attempted to obtain my essays from the Virginia Board of Bar Examiners.  I was told repeatedly that once the results are released they are final and there is no possibility of review.  At the end of November I filed an emergency motion in Fairfax Circuit Court seeking a mandamus, a judicial command order, under the Administrative Process Act, to compel the release of the essays.  Senior Judge Leslie Alden heard the arguments for a half hour.  An assistant attorney general, who came up from Richmond, did not at that time dispute that there was a software failure.  When asked by the judge, "what remedy is available to aggrieved bar examinees," she stated:  "that is an interesting question."  The judge then took the case under advisement and requested I file a supplemental brief on the applicability of the Administrative Process Act (see enclosed).  It was discovered that the Virginia Board of Bar Examiners is among a list of exempted entities from the general application of the APA (by the Code of Virginia, as one of three agencies of the Virginia Supreme Court).  Unlike practically all other state agencies, therefore, the Fairfax Circuit Court had no jurisdiction.

Note that in her decision (enclosed), Judge Alden found that while there is no explicit statutory authority to review the decisions of the Board, the Virginia Supreme Court retains such inherent authority and that I should therefore invoke the original jurisdiction of the Virginia Supreme Court in this case.  Note also that the judge, in footnote 2, states:  "the Court does not reach the issue of whether the Board has abused its discretion by adopting a policy that it can never release bar exam answers to applicants."

In the opinion the court suggests:  "To the extent the Petitioner seeks a mandamus ordering the Board to release his bar exam answers, the Petitioner should invoke the original jurisdiction of the Supreme Court.  Va. Const. art. VI, Section 1." 


MAY 2009 VIRGINIA SUPREME COURT (Chief Justice Leroy Hassell).  As instructed by the lower court, I brought the mandamus action to the Virginia Supreme Court under Va. Const. art. VI, Section 1.  Along with the complaint, I included my sworn affidavit as to the details of the software failure that affected my test as well as observations I made about other applicants who experienced the same difficulties saving the test essays.  I also disclosed how 24 applicants were found to have had essays mispositioned in the software system from the morning session as evidenced by an announcement made at the test site.  In addition, in the complaint I referenced the case of an applicant in New York who experienced the same software symptoms that I did who was able to prove misgrading of his test essays only once he was able to obtain his essays. 

The court simply dismissed the petition altogether.  What little the court did say was mandamus does not lie to compel a discretionary act (which it does, in the case of abuse of discretion, which was clearly a part of the claim).  In a petition for rehearing, I claimed that there was no other forum to hear the case and by not hearing the case (or such cases) the court is propounding a due process violation of the Federal Constitution.  Still, the court denied rehearing, again without explanation.


AUGUST 2010   U.S. DISTRICT COURT FOR THE EASTERN DISTRICT OF VIRGINIA.  Judge Robert Payne.  Because the Virginia Supreme Court declined to exercise its own inherent authority over the bar exam, I was essentially left without redress in the Commonwealth of Virginia.  I then brought the case to federal district court as a violation of due process on the part of the Virginia Supreme Court and its Board of Bar Examiners.  The case precedent for jurisdiction here is Rogers v. Supreme Court of Virginia, 772 F. 2d 900 (4th Cir. 1985) ("we think that, when that court (Virginia Supreme Court) reviews or declines to review a decision of the bar examiners as to an individual examinee's status, the court is acting as an administrative agency, rather than as a court of appeals.  Therefore, we find it appropriate to determine whether or not Rogers' allegations concerning the review procedures of the court and the board make out a viable claim.").  See also Richardson v. McFadden, 540 F.2d 744, 750 (4th Cir. 1976) ("It is beyond question that the bar examiners are subject to the requirements of due process and equal protection in the conduct of their duties") and Whitfield v. Illinois Board of Bar Exmnrs., 504 F.2d 474, 478 (1977) ("the due process clause requires the State to employ fair procedures in processing applications for the admission to the bar and, therefore, that an applicant who has failed the bar exam is entitled to some procedural protections"). 

I obtained a leading computer forensic expert who submitted a brief to the court where he was able to already form certain conclusions which corroborated that my concerns were indeed justified and that the descriptions of what occurred in New York's software failure matched what is being observed in Virginia.  See attached expert report of Stephen Castell PhD.  The expert also concluded that the National Conference of Bar Examiners has already developed an "obvious and sensible remedy" to use for situations that come up with the computer based test (see last sentence of Stephen Castell's report).  It also characterizes my case, in Paragraph 31 of the expert report, as a "textbook case" and that the Board's unwritten policy of nondisclosure "is an unusual way to design a system and immediately gives rise to concerns over validity and completeness checking."

In the complaint, Paragraph 17, I allege that the unwritten policy of nondisclosure of test essays has the effect of rendering the right of every bar exam applicant to petition the Supreme Court of Virginia defective and illusory.  That is because of the basic requirement that one "plead with particularity" so as to make out a cognizable claim.  Without such particularity, cases are dismissed by the Virginia Supreme Court (and all courts) on a routine basis.  See Application of Peterson, 459 P.2d 703 (Alaska 1969), where the Alaska Supreme Court called the board of bar examiners' position a "logical hiatus" in that it required a bar examinee to demonstrate error without affording him a device in which to locate that error.  The essays are known as "sine qua non" evidence in this instance- evidence without which there exists no case.

In my prayer for relief, I ask for no monetary compensation whatsoever.  I merely ask that the unwritten policy of nondisclosure be struck down as unconstitutional and that my essays be released to me at once. 

Defendant's Answer.  The Board's Answer to the Complaint (enclosed) admits to the unwritten policy of nondisclosure.  See Paragraph 13.  It furthermore confirms that it has been applied in this case and has been applied uniformly to all applicants.  In chambers, the assistant attorney general was corrected by Judge Payne when she misstated that it has always been this way, because he said it was not when he took the exam.

Court's Decision (see enclosed).  Without addressing the merits, the court dismissed the case on the premise that I was challenging the Board of Bar Examiners' unwritten policy with respect to my own exam as opposed to all applicants.  In fact, I was doing both.  As the prayer for relief states first and foremost:  "That this Court take the necessary steps to protect the rights of all future applicants to the Virginia Bar by declaring Policy of Nondisclosure and Policy of Finality unconstitutional."

Court of Appeals for the Fourth Circuit (see enclosed). The decision of the lower court is sustained without giving any reasons.  I have enclosed my brief.  The attorney general's office did not bother to respond, other than with a paragraph or so.


JUNE 2011 U.S. DISTRICT COURT FOR THE EASTERN DISTRICT OF VIRGINIA.  Judge Henry Hudson.  Since the previous case was dismissed for challenging the policy of nondisclosure with respect to my own exam, this time I brought forth a new action simply challenging the policy on its face as unconstitutional and seeking no relief for myself whatsoever.  See the enclosed complaint.  By this point, I had waived in my multiple choice score from Virginia (not the essay portion) into D.C. and took only the essay test in D.C.  D.C. runs a fully transparent system, and with a 60% failure rate, I still was able to pass on my first attempt.  Virginia is a mere 15% fail rate for first time takers.

Board Threatens Sanctions.  When this second case was filed, which becomes a part of the record, the attorney general's office claimed that I was bringing "vexatious lawsuits" and sought sanctions against my D.C. license.  Previously, this threat was made as an attempt to get me to withdraw the case.  And my response letter (see response to the sanctions motion, attached) was sent to the assistant attorney general as well as personal and confidential to Attorney General Ken Cuccinelli, explaining my reasons why this case had to go forward.  Even though this assertion as to "vexatiousness" was completely without basis, this issue was litigated and the court found in my favor and denied the motion.  Therefore, the case was able to proceed. 

Court's Decision.  The court issued a 9-page opinion.  At the heart of its rationale, found on the bottom of p. 7, is the idea that federal courts will not review due process or equal protection cases unless they involve cases of discrimination.  In other words, racial discrimination cases are the only ones they will review.  Other cases cited to in my brief that show federal review is not and cannot be just limited to discriminatory cases were simply ignored and not addressed in the opinion.

Ironically, as opposed to the prior district court decision which concluded it had no jurisdiction because I was applying the facts too much to my own case, this district court concluded no jurisdiction on exactly the opposite ground:  "In crafting his complaint, Plaintiff seeks relief on behalf of all future bar applicants and appears to distance himself from the group of individuals directly affected by the statute."  (p. 6 of the opinion).  Ignoring my primary contention that a policy withholding essays in a disputed essay exam prevents a case from going forward to the Virginia Supreme Court, the court lackadaisically states that applicants can petition the Virginia Supreme Court or take the bar exam again.  Without transparency, and given my personal experience with the Board, it should be quite obvious why I would never take the Virginia bar exam again without an appropriate change in policy.  This forced me to go elsewhere to gain admission to the Bar.

Court of Appeals for the Fourth Circuit (enclosed).  Sustained the lower court's decision, without explanation.


PETITION FOR CERTIORARI BEFORE THE U.S. SUPREME COURT (enclosed booklet).  Denied October 1, 2012.  The Supreme Court is careful to note that a denial of cert is by no means an expression on the merits of the case.  All legal remedies at the state and federal level have therefore been exhausted.  Despite the evidence and witness testimony available, the case never was heard on its merits.   

VIRGINIA BAR EXAM: LACK OF TRANSPARENCY

Update 33.  The courts, having punted this issue over to the General Assembly, have demonstrated the flaw in the procedural protections afforded to applicants to the Virginia Bar.  With the enclosed letter (see below) I have notified a select group of state delegates and senators who may be willing to sponsor legislation that will bring Virginia up to speed with the other states in terms of transparency.  The Board of Bar Examiners is on record in federal court as admitting to having an unwritten rule, unbeknownst to the candidates, that prevents any applicant from obtaining their essay answers, a rule they admit was applied in this case.  I am informed by a witness to the latest exam in 2014 that the Board's software failures persist and continue to affect significant numbers of applicants at every succeeding exam.  This is what happens when people, including judges, look the other way.

I have since taken the D.C. and Maryland bar exams and passed on the first attempt.  Both systems operate transparently.  I have decided not to take the Virginia Bar Exam again because I have lost confidence in the integrity of the system.


Dear Delegate/Senator:

I am writing to you about a proposal that I have enclosed which would remedy a fault that exists in the current procedures of the Virginia Board of Bar Examiners.  Let me first say that the Board is an administrative agency with both judicial and delegated legislative powers.  And while there is support at the local level in northern Virginia for this proposal, this will require your office's support to succeed because the Board of Bar Examiners has refused to cooperate in making this adjustment towards transparency.  I am an attorney and a magistrate in Fairfax and, based on what I have observed and experienced, there is no doubt that this proposal is what is necessary to protect the applicants and correct an ongoing problem with the new computer based bar exam.  I attempted to litigate this issue, but the Virginia state courts and federal courts have decided that it is essentially a political question for the General Assembly and decided not to hear the merits of the case.

SOFTWARE FAILURES AT THE VIRGINIA BAR EXAM

Virginia's bar exam is now being administered on computer software provided to the applicants by the Board of Bar Examiners.  The Board contracts with an out-of-state software company to provide this software so they can administer their test for over a thousand applicants every year.  When I took the Virginia Bar Exam, I was one of a significant number of applicants who experienced a software system failure that occurred at the moment when the essay answers are to be saved and submitted.  Members of the Board's technical team attempted to work on the problem but ultimately said the system had to be rebooted (abruptly turned off and then turned back on).  One of the female applicants was in tears when this happened to her.  An announcement was made that over the lunch break they had already discovered that 24 applicants had answers misplaced in the system from the morning session.  This was exactly what happened in New York the year before:  see attached, New York Law Journal "Software Snafus Upset Test Takers," July 26, 2007.  I was then forced, without being able to see and verify that the answers were intact, to transfer the potentially corrupted data onto a USB thumb drive and hand it in to the graders.  Upon receiving my essay score months later, I noticed a significant scoring discrepancy between the reported score and how I had performed on that day.  No other information was disclosed.  Of course, I immediately requested to obtain my essays.  In response to my request, however, I was told there existed no corrective process and they had an unwritten rule, applied uniformly, that no applicant can obtain their essays for any reason.  Even under these circumstances, where I explained that I had a job lined up at the prosecutor's office and produced a sworn affidavit saying exactly what had occurred at the test site, the secretary of the board refused to cooperate.  Moreover, I was able to obtain a 20-page expert report from a leading computer forensics expert who concluded that the fault lay not with my computer but with the board's software system.  As a result, I was given no redress and consequently lost a career position at the Arlington Commonwealth Attorney's Office.

SHAM REMEDY

The only remedy offered to me was a sham remedy wherein the board agreed to examine only what was on the USB thumb drive and do a comparison with what was printed off from the USB drive for the graders to grade (recall that the data was transferred from the software to the USB drive only after the software system failure . . . at no time was I permitted to confirm if the answers were still intact).  The computer expert, and three other experts I interviewed, have been unequivocal in their conclusions that this is no remedy at all and does not address the underlying issue, namely whether the data that was saved on the USB drive in the first place was corrupted.  Therefore, a comparison exercise like this accomplishes nothing because a copy of already corrupted data will necessarily be a match; nevertheless, it is still corrupted data.  This is misleading.  Therefore, there is no proper way to go about this other than to allow the applicant to obtain his/her essays and determine if what was graded includes everything that was written.  That is what the National Conference of Bar Examiners recommends for the computer-based test, and that is how it was handled in New York.  All of this was explained to a federal judge but, as I mentioned, the judge avoided the question, ruling this was essentially a political question for Virginia's legislature.

SAME PROBLEMS CONTINUE TO THE PRESENT DAY

This occurred in 2008, and I have a witness who confirms that he observed that the board's software failures continue to affect the applicants up to the most recent examination in 2014.  Furthermore, this is a problem that is happening nationwide since the advent of the computer based test in 2005.  For example, in New York's 2007 computer-based examination, it is well documented that as many as 47 applicants had essays that were either missing or cut short by the software after experiencing the same symptoms that I and others experienced in Virginia.  See enclosed Press Release, New York Board of Law Examiners (Nov. 15, 2007) (available on archive at http://web.archive.org/web/20071123015445/www.nybarexam.org/press.htm) (specifically paragraph 5 which reads "one or more of the essay answers for 47 candidates could not be recovered").  These applicants were allowed to obtain their essays; and once they did, it forced a corrective process that was not previously in existence.  In the case of Eric Zeni, this made the difference between passing and failing.  See enclosed Eric Zeni's account in the New York Personal Injury Law Blog.  Had Eric Zeni been in Virginia when this happened, there would have been absolutely no recourse.  This cannot stand, and it violates the due process rights of the applicants who expend an enormous amount of time and resources into this test and have secured positions contingent on their passing. 

LEGISLATIVE HISTORY

Allow me to provide you some background on the legislative history of this issue.  In 1919 the General Assembly enacted Va. Code Section 54.1-3929 which requires the Board of Bar Examiners to preserve an individual's examination papers for a period no less than one year following an examination.  This provision was relied upon by applicants and their attorneys whenever a dispute would arise.  This important protection to the rights of the applicant and the integrity of the system as a whole has been effectively nullified sometime in the early 1970's when the Board put into effect an unwritten policy, unbeknownst to the candidates, that prevents applicants from being able to obtain or review their essays.  This policy is being applied uniformly without exception, a fact I have the Board as having admitted to on the record in federal court.  Prior to that, applicants could obtain their essays.  This is a fact that I know because U.S. District Court Judge Robert Payne in Richmond replied that he could have gotten his answers when the Board attempted to say it has always been this way.  Since 2005 the Board has been administering a computer-based bar exam for the essays, a condition that demands a change in policy at this time.

NATIONAL MOVEMENT TOWARDS TRANSPARENCY

In light of the national transition to computer-based testing, 44 of the states now afford their applicants an opportunity to review or obtain their essays, for any reason.  See enclosed chart, "Transparency Policies of State Law Examination Offices."  I hope that you will see that it is time for Virginia to acknowledge the new realities of computer-based testing and the need for transparency in the system.

The vast majority of the other state boards of bar examiners have rules that protect their applicants.  In Virginia, the General Assembly sets laws from which the Board must derive its rules.  Va. Code Section 54.1-3929 mentioned above is an example of one such law.  The only way for this problem to be solved is for the General Assembly to act, because the Board is insistent on resisting this change.

ABOUT ME

To provide you with a brief background on myself, I went to William & Mary Law School (Class of 2008) as well as the College of William & Mary after transferring from the U.S. Military Academy at West Point.  I have since taken the bar exam in the District of Columbia and Maryland, having passed both on the first attempt and have been admitted to practice in the federal courts and pending admission in the U.S. Supreme Court.  However, I have not taken the Virginia Bar Exam again because, with an unwritten policy like this one in place, I have lost confidence in the integrity of the system.  For more background on my story, I have published an article along with the expert I consulted in the Computer Law & Security Review (2013) (attached). 

PROPOSAL

Please see my attached proposal, which includes language taken from other states which Virginia can look to for guidance to solve this problem.  The Virginia Board of Bar Examiners is a state administrative agency with both judicial and delegated legislative powers. 

This nonpartisan issue affects not just candidates for the Virginia Bar but by extension it affects the people of Virginia, causing well qualified lawyers trained in Virginia to leave the Commonwealth and seek admission elsewhere.  Most legislation relating to the Virginia Bar Exam will be sought by the Board, but this one is for protecting the applicants from the Board's iniquitous unwritten policy of nondisclosure of test essays.  I will certainly make myself available to provide a statement or sworn testimony before any committees as necessary.  Furthermore, the expert report and his contact information is available upon request.

Thank you for your time, and I look forward to working with you on this important matter affecting the Commonwealth. 

Regards,

Jonathan Bolls

Virginia Bar Exam: Lack of Transparency

Update (32):  The following was just published in this quarter's edition of Computer Law & Security Review (Volume 29, Issue 4 (2013), p. 446-449, Computer Based Testing, available at www.sciencedirect.com :

"Computer based testing - Bolls v. W. Scott Street, Sec'y of Va. Bd. of Bar Exmnrs., August 11, 2009, case no. 090915"

By

Jonathan Bolls and Stephen Castell PhD

ABSTRACT

The most significant change to law licensing in the United States is the recent reliance on computer-based testing for the bar exam in all of the fifty states.  Virginia, for example, has been providing the computer-based test on a wide scale since 2005.  New Jersey was the very first, in 2001.  Since then, almost all of the states are now administering a computer-based test.  The following comment piece is meant to inform the computer and technology law community how the use of testing software must necessarily be accompanied by additional procedural protections and corrective processes that may not have existed previously.  Chief among these is the availability of discovery of the item in question, namely in this case the essays in a computer-based bar exam, which constitutes evidence that is essential in vindicating one's rights when a software failure occurs.

ARTICLE

      In July of 2008 I (Jonathan Bolls) took the essay portion of the Virginia Bar Exam on special testing software provided by the bar examiners.  At the conclusion of the morning session, the software froze while in the process of saving the essays.  There was a team of technicians on standby and the candidates were instructed to seek their assistance if anything went wrong with the software.  At the conclusion of the test, when the proctor asked if there were any problems, a significant number of hands went up including my own, visibly overwhelming the full team of technicians on standby.  It took a full 15 min before one could come around to assist me.  He worked on my computer hands-on and, when that didn't work, he instructed me to reboot and try saving the answers again.  So I did.  But after the reboot I was not able to see any of the content of what supposedly was being saved.  Then I had no choice but to transfer the file to a USB drive which was then handed in for grading.  That afternoon, prior to the start of the second session, the candidates were informed by loudspeaker that over the lunch hour they discovered that 24 candidates had essays which were "misplaced" in the USB drives.  These candidates did not know who they were and would receive further instructions in their afternoon test booklets on how to rearrange their essays properly.  After the afternoon session, I again experienced the same software malfunction that happened in the morning requiring hands-on technical assistance, which ultimately failed, and had to reboot.  Again, I was by far not the only one.

     Months later I received my results and became aware of a major scoring discrepancy in my essay score.  With a career position hanging in the balance, I immediately contacted the Board of Bar Examiners and requested that copies of my essays be sent to me.  My request was denied at once by the secretary of the board simply because, as he stated, there is no review or appeal process available and the results are final.  Furthermore, there exists an unwritten rule that no applicant can obtain their essays.

     I began to do some research and it was not long before I discovered that software system malfunctions are happening throughout the different state bar exams of the country and, in fact, data has been found missing, misplaced, or otherwise altered in some way.  Indeed, it has been proven that misgrading happened as a result.  Most interestingly, perhaps, is the fact that these affected applicants had been previously reassured (just as I was) that their answers were intact.  It was not until they received their essays as of right (which the vast majority of the states allow) where applicants became so outraged that they forced a corrective process not previously in existence.

     With a career position at a district attorney's office on the line, I had no choice but to bring a mandamus action (judicial command order) in Fairfax, Virginia to force the state agency to release the essays.  At the hearing, the local judge asked the attorney representing the state what process is available to aggrieved bar applicants, to which the answer was "that is an interesting question, Your Honor."  This set the case on a constitutional detour that I would never have expected to be on so soon out of law school.

     The case was taken under advisement, and while I was informed by a judicial law clerk that the judge was persuaded that the essays had to be released, it was ultimately determined that since the Board is an arm of the Virginia Supreme Court, only that court could issue such an order.

     Before taking the case to the Virginia Supreme Court, I conducted some research of my own, and what I discovered was interesting and troubling.  The year before (2007), applicants who sat for the New York bar exam complained of software glitches and system reboots in the process of saving the test essays, basically the same symptoms that I and numerous others in Virginia experienced.  See New York Law Journal, available at http://www.abajournal.com/news/ny_bar_exam_marred_by_glitch/print/   One key difference between New York and Virginia, however, is that applicants in New York have a right to obtain their essays.

     This right proved crucial for 47 applicants who discovered portions of their essays went missing, were duplicated, or otherwise altered in some way by the software system failure.  Initially, it was the testing board's position that there was no cause for concern, that all the essays were received in their original and complete form.  Even after the board released a press release about the 47 (New York Bd. of Law Exmnrs, available at (5th Paragraph Down) http://web.archive.org/web/20071123015445/www.nybarexam.org/press.htm ), we know of at least one applicant in particular (Eric Zeni) who demonstrated that the 47 was not all that were affected (New York Personal Injury Law Blog, available at http://www.newyorkpersonalinjuryattorneyblog.com/2008/02/i-passed-the-new-york-bar-exam.html ). 

     Eric Zeni, for example, demonstrated that two essays, not just one, were cut short by the software.  He has told me privately that the board actually tried to place the blame on him, as if it was somehow his fault their software wasn't working properly.  In the end, he was able to prove two of his essays were impacted by the software failure and should not have been graded because they were not reflective of his performance on the test.  After a proper review, his failing grade was therefore changed to a passing grade.  New York is therefore an example of how transparency upheld the integrity of the system- despite some level of inconvenience to the bar examiners themselves.

     Since 2007, the National Conference of Bar Examiners has created a remedy in response to what happened in New York, which includes identifying the essay portions that were affected and then grading the balance of the examination.  See New York Board of Law Examiners' press release, Par. 5, available at http://web.archive.org/web/20071123015445/www.nybarexam.org/press.htm

     Prior to bringing the case to the Virginia Supreme Court, I called every state board of bar examiners' office and inquired whether applicants have rights to the essays following an exam.  42 of the 50 states said they do (with Colorado most recently changing its policy to allow this right, increasing the number to 43).  Virginia stood as one of a small minority to operate a computer-based test with an unwritten rule that no applicant can obtain their essays, even in the event of a software system malfunction.  The other states that are still doing this include Arkansas, Georgia, Hawaii, Missouri, South Carolina, South Dakota, all of which are currently administering computer-based tests.

     One right that all applicants in each of the States have in common is they may present their grievance to their state supreme court, which has inherent authority over bar admissions.  As in the case when any legal claim is brought, evidence is needed to show injury.  In this case, only the release of the essays could do that.  The Board merely cited to an unwritten rule instituting in the early 1970's that no applicant can obtain their essays.

     I therefore had no choice but to bring a mandamus action, seeking a command order of the state supreme court to compel a public official to do his duty under an abuse of discretion claim.  I have confirmed that in my case no guidelines for discretion were followed.  No investigation of the software failure took place whatsoever.  It simply came down to a blanket rule that no applicant can obtain their essays.  With my sworn affidavit, Eric Zeni's testimony for what happened in New York just the year prior, the existence of a remedy developed by the National Conference of Bar Examiners, and the presence of a witness who can confirm my observations at the test site, the court still refused to hear the case or order the release of the test essays.  That being the highest court in the state, I was left without recourse and without being able to have my day in court.

     Because of the failure of Virginia's judicial system to provide any forum for handling software claims of bar examinees, I decided to bring a constitutional challenge in federal court to the Virginia state board's policy preventing applicants from obtaining their essays even following a system software failure.  Such a rule runs directly afoul of the due process protections of the applicants, particularly given the recent changed circumstances of the computer-based test.  The relief I was requesting was simply the release of test essays.  No request for damages or monetary compensation was included in state or federal court as I was seeking merely to vindicate my own rights and fix the obvious problem for all future examinees.

     First, I needed a computer forensics expert to look at this issue and provide the science behind the common sense as to why this rule is no longer appropriate in the environment of computer-based testing.  I had the great fortune to come across Dr. Stephen Castell, an IT Consultant of the Year Professional Award Medalist of the British Computer Society (http://www.CastellConsulting.com). Over the course of his thirty-five year career, Dr. Castell has become a leading international expert in his field and has testified in the English High Court, the Sydney Commercial Court, and American Federal Courts, for both plaintiff and defense, and for many major clients worldwide.  He has acted in both the longest (GEC-Marconi v LFCDA, 1991-1992) and largest (AirTours v EDS, 2001; total claims $250M pounds sterling) computer software development contract lawsuits heard in the English High Court.  After hearing about this case, he was kind enough to submit a 20-page report pro bono in support of the case. 

     The federal court had all the evidence that the Supreme Court of Virginia had with the added benefit of Dr. Castell's report in which he was able to form certain preliminary conclusions and opinions based on what happened at the test site.

     First, he was able to conclude that there was no way for a proper review to take place without including the applicant, the key creator of the data.  That is because, as he states, the need for a reboot under these circumstances is "a very real cause for concern," and "the nature of computer-based testing systems is that there will inevitably be instances where cooperation with the applicant, the key creator of the critical data involved, is necessary, if not vital . . . [i]t is as important also to remember that Mr. Bolls was not the only one affected, and his case may prove helpful to improving the accuracy, integrity, and reliability of systems if, like Eric Zeni, corruption of the essay responses is discovered." 

     Secondly, he was able to conclude that the Board's offer to compare what was saved on the USB drive for hand-in to the graders with what was saved in encrypted format on my computer was just a sham remedy because the software system failure took place before the data was even transferred to the USB drive.  Therefore, as he states, "[T]his can only merely confirm that the file was written by Mr. Bolls using the Exam4 software.  In no way does this pose a remedy to, or even a sensible investigation of, any potential corruption problem caused by the Exam4 software itself, or through other (temporary or permanent) system fault, for which Mr. Bolls could not have been responsible." 

     Third, he affirmed the usefulness of the remedy as developed by the National Conference of Bar Examiners, which is entirely foreclosed to applicants in Virginia until there exists a means for obtaining the essays:  "If, as in my view, there were in Virginia to be the reasonable, and technically sound, policy in place for applicants to obtain and inspect the essays, then, if such portions were thereby identified, it seems to me that there would then be an obvious and sensible remedy, viz, to employ the alternative grading methodology developed by the National Conference of Bar Examiners."  It is important to note that no expert was brought forward by the Board of Bar Examiners; in fact, I had found two other experts who came to the same conclusions that Dr. Castell did. 

     The federal court nevertheless dismissed the case without a hearing or determination of the merits on the grounds that I was challenging the agency's rule as applied to my own exam and not as a rule applied to all.  This actually wasn't true as I made it very clear in my complaint to the court that besides relief for myself (in the form of releasing the test essays) I was also challenging the rule on its face and sought in my relief to have the rule struck down for future applicants who will have to undergo a computer-based test ("That this Court take the necessary steps to protect the rights of all future applicants to the Virginia Bar by declaring Policy of Nondisclosure and Policy of Finality unconstitutional").  The appellate court affirmed without explanation.

     While preparing for the next step in the case, I decided to take the bar exam in another jurisdiction, and was able to successfully pass the Washington, D.C. bar on my first attempt (handwritten not typed).  I have taken a job as a magistrate judge in Fairfax, Virginia, a position that is appointed by the Supreme Court of Virginia.

     It was becoming clear something was very wrong with how the judicial system in Virginia was approaching software disputes with the Board of Bar Examiners.  Either they did not want to take the time to understand the science and hear the expert or they were simply hostile to the issue.  There is no doubt that with the evidence presented, this case should never have been dismissed without a proper hearing. 

     Taking the cue from the last decision, I brought the case again in federal court, this time not asking for relief for me at all, but rather a declaratory judgment that the rule that no applicant can obtain their essays and the corresponding state law preventing Freedom of Information Act requests for test essays was unconstitutional on its face in light of the recent transition to computer-based testing in Virginia.  But that judge too went out of his way to deny jurisdiction, this time extending just the opposite reasoning to deny a hearing on its merits.  This time, the court said that I lacked standing.  In other words, it was the court's opinion that I was not applying the case enough to my own injury and so could not bring it on behalf of all the others.  It was a true Catch-22.  Again, this was based on a false premise but nevertheless the only way for the court to ensure the evidence and witnesses were not heard in open court.

     Because of the high importance of the integrity of bar admissions and the novel issues at play with technology, due process, and movement towards more transparent policies nationwide to protect the applicants, I brought a petition before the U.S. Supreme Court to correct the antequated policies of the 7 hold-out states, including Virginia, where applicants have no recourse when a software malfunction affects their test essays.  Certiorari was not granted for that.  Unfortunately, the Court can only hear a mere 2% of its cases, though it is widely known that is only a small fraction of the many cases that have actual merit.

     So we see the Judiciary clearly has its limitations when it comes to software related disputes.  I have no doubt that something went wrong with my essays, and the Board's having fought tooth and nail to keep me from seeing them, for years, says something in and of itself.  To this day, the essays remain saved on an encrypted file held on my laptop, the key to which remains in the hands of the Board.  Had my test been administered in Alaska, for example, the essays would have been sent to me upon request for a charge of $10; in Florida it would have been $50. 

     As Dr. Castell put it well, under these circumstances where a dispute presents itself, the essays themselves represent sine qua non evidence i.e. no chance of bringing a claim that could withstand a motion to dismiss for lack of evidence.  A healthy judicial system must be willing to address technology and people's rights, even if it is not a high dollar case, because it goes to the core of individual liberty.  Short of this, rights will be trampled by those who take advantage of the courts' limited understanding of technology and unwillingness to deal with the obvious implications for citizens' rights.  This will inevitably lead to an unjust result for the individual, which in this case impacts not only the candidates but the public at large.

     There are many benefits to technology.  But as technology increases, transparency must increase in like measure even if it means additional procedural safeguards are put in place.  That is the only way to protect the rights of the individual and our freedom in today's Information Age (*footnote).

*Footnote:  Particularly when evidence derived from Information Technology may arguably be 'ontologically unreliable', as noted earlier in this Journal by Dr. Castell - see for example 'A computer of the simplest kind...', Letter, page 158, [1994] 10 CLSR, May - June 1994, and its appended Footnotes.

VIRGINIA BAR EXAM: LACK OF TRANSPARENCY

Update (31):  I have written an article, along with Stephen Castell PhD, the expert in this case, that was peer reviewed and published in this quarter's edition of the Computer Law & Security Review (Volume 29, Issue 4 (2013), p. 446-449, Computer Based Testing).  I will post it to this site shortly. 

Dr. Castell has become a leading international expert in his field and has testified in American federal courts for both plaintiff and defense, and for many major clients worldwide. 

Update (30):  May 11, 2013.  As a brief update, steps have recently been taken to refer this issue to the legislature of the Commonwealth of Virginia.  At the start, when I was granted an emergency hearing, it quickly became clear that this matter would not be resolved on an emergency basis, which would have saved that career position by simply allowing me to obtain my essays.  The question then became, what position do the courts take on such a matter?  I then began to embark on a course to find out.  Who else could take this course but someone like myself with such a unique position to have standing?  The judicial system, in order to make its rulings on constitutionality, must have before it an injured party to demonstrate why there is a constitutional violation (in this case a due process violation).  The courts are now on record as having dismissed the case without hearing it on its merits.

VIRGINIA BAR EXAM: LACK OF TRANSPARENCY

Update (29): On October 1, 2012 the Supreme Court of the United States denied certiorari on this case's constitutional challenge to Va. Code Section 54.1-108(1) which precludes FOIA requests for bar exam essays (certiorari brief included below). The Supreme Court is careful to note that denials of cert are not an expression on the merits. Indeed, it only has the time to hear a mere 1% of the cases. This was a paid case that sought no monetary or individual relief for me; rather, it was meant to draw the courts' attention to a procedural irregularity that uniquely affects Virginia bar examinees (and the examinees of 6 other states). Interestingly, after exhausting the state and federal courts, no court, including the Virginia Supreme Court, has passed upon the merits of the case. The case charted its own path as one judge after another passed the baton, each one concluding it lacks jurisdiction for one reason or another.

The Virginia Supreme Court, which is the court that should have heard my individual case, preliminarily concluded it lacks the authority to mandamus (command order) the release of the essays, even in the context of a malfunction in the Board of Bar Examiner software system affecting a significant number of people, because the underlying decision whether or not to release them was a discretionary decision by a state official. However, the Virginia Supreme Court was presented with information that the Board has made it an unwritten policy that no one, including those who fall prey to its software failures,can obtain their essays. I have argued extensively that to apply a blanket rule like that is to ignore what we all know in our common sense is totally improper in any computer-based test. The Virginia Supreme Court and later the federal courts were also shown another Virginia law (Va. Code Section 54.1-3929) that requires the Board to preserve the essays for one year, a law that has been used in Virginia prior to the enactment of Va. Code Section 54.1-108(1), is still in effect today, and is used in other states to allow an applicant (or his/her representative) to obtain their essays following the exam.

Aside from a leading computer forensics expert's conclusions regarding the software failure and his citation to discoverable evidence, all the courts were directed to what appears to be the heart of the problem in Virginia: the Board, as an agency of the Virginia Supreme Court, is exempt from APA (Administrative Process Act) review by a state circuit court. And the only court that has supervisory authority over the Board of Bar Examiners is the Virginia Supreme Court. That court, in 2009, was informed in the initial pleadings that a leading computer forensics expert was retained and has been able to form significant conclusions as to the nature of the Board's software system failure and its policy of nondisclosure. Nevertheless, the court refused to hear the expert and immediately denied jurisdiction to compel the release of the essays because the Board has discretion on whether or not to release an individual's essays. This is a "catch-22" for aggrieved applicants and a classic due process dilemma, because the case law is clear that no claim, without the evidence to support it (i.e. the essays) would be able to survive dismissal if brought before the Virginia Supreme Court. As a result, I am informed that the software problems persist and there is still no recourse available for software system malfunctions at the Virginia Bar Exam. My individual case demonstrates this having been first argued orally in Fairfax Circuit Court in December 2008. While I am informed that court was persuaded of the injustice, it simply lacked the jurisdiction to do anything about it. When the judge pointedly asked the Virginia Attorney General's office at oral argument what process is available to an aggrieved bar examinee, the response was: "That is an interesting question." That immediately set my case on a journey of its own, which was followed by a constitutional challenge in federal court to Va. Code Section 54.1-108(1) which prevents all of these cases from being heard by the Virginia Supreme Court.

All of this begs the question, what is the role of the courts? I believe that courts should be courts of justice. There are some who believe otherwise, that the courts exist for the sole purpose of processing claims. If we fall into the thinking of the latter, then the courts will likely assign higher priority to high dollar cases and leave less consideration to cases like this one where a state law has been identified as violating individual rights and leaving a particular state agency without effective review. In a civilized society, the rights of the individual must be upheld by the courts.

The experience of having gone before so many state courts and federal courts in just a few years following law school has been both educational and rewarding. I am now intimately familiar with the Supreme Court Practice treatise, due process law, and bar admissions law throughout the country. The issue is certainly not going away any time soon, for example, the expert report still stands.

Clearly, the judiciary has its limitations. And where the judiciary fails, it is up to the People to decide whether a change in the law is appropriate. I will now consider appropriate next steps in addressing this issue. Please see, below, a copy of the certiorari petition:

                             QUESTIONS PRESENTED

Applicants who experience a software system malfunction at the Virginia Bar Exam have a right to bring claims to the Virginia Supreme Court. However, the Board follows a policy, uniformly applied, that no applicant can obtain their essays, a policy premised on Va. Code §54.1-108(1). Forty-Three (43) States plus the District of Columbia have adopted policies to allow their applicants to obtain or review their essays in light of the recent transition to computer-based testing, a number that is on the rise. Well more than half of bar applicants nationwide now rely on software provided by their testing boards.

1. Whether the changed circumstance of the transition to computer-based testing for the bar exam nationwide coupled with the overwhelming trend among the States allowing applicants to obtain their essays allows for a valid challenge under the Due Process Clause of the Fourteenth Amendment to a state policy that denies such rights.

2. Whether a bar applicant who has a software dispute with the board has standing to bring a prima facie constitutional challenge to a state law that bars applicants from obtaining their bar exam essays when the essays represent the only evidence that could be used in such disputes to petition the state supreme court under its original jurisdiction.

DECISIONS BELOW

The dismissal of the U.S. District Court for the Eastern District of Virginia is unreported and reprinted in the Appendix. Summary affirmance by the U.S. Court of Appeals for the Fourth Circuit is also reprinted in the Appendix.

JURISDICTION

The dismissal order of the Fourth Circuit was issued on February 13, 2012. This Court has jurisdiction under Article III, Section 2 of the Constitution. The federal question arises under the Due Process Clause of the Fourteenth Amendment of the Federal Constitution.

STATEMENT OF THE CASE

As the States have made the transition to computer-based testing for the written portion of the bar exam, there remain seven state bars that have kept policies in place to prevent applicants from being able to obtain their written essays following the exam (Appendix C, Chart, Transparency Policies of State Law Examiner Offices). Virginia is one such state. After being one of a significant number of applicants who experienced a software system malfunction during the Virginia Bar Exam, Petitioner noticed a scoring discrepancy in the results and sought to obtain his essays. After being prevented from doing so, notwithstanding a Virginia law that requires the Board to preserve the essays for one year (App. F), he sought relief from the Virginia Supreme Court pursuant to its inherent authority over the bar exam. That court, however, ruled that it did not have the authority to release test essays. Without reviewing the merits or hearing Petitioner’s expert witness, the court simply dismissed stating as follows:

"On consideration of this case, the Court is of opinion that mandamus does not lie to
compel the performance of a discretionary act and the writ of mandamus should not issue as prayed for. "

Bolls v. W. Scott Street, Sec’y of Va. Bd. of Bar Exmnrs., August 11, 2009, case no. 090915.

Consequently, his petition never could be heard on its merits and it remains as yet undetermined the extent of the affect the software malfunction had on his test essays and those of the others who sought review.

On June 24, 2011 he brought a prima facie constitutional challenge in the U.S. District Court for the Eastern District of Virginia to Va. Code §54.1-108(1) (reprinted in Appendix F), which prevents bar examinees whose essay scores are in dispute from obtaining their essays under the Virginia Freedom of Information Act. Since the recent transition to computer-based testing in Virginia in 2005, applicants continue to experience software system malfunctions requiring hands-on technical assistance and a system reboot during the saving stage of the test. The Respondent admits this occurs at every examination sitting. Despite the fact that the symptoms observed are consistent with proven data loss and misgrading in other state bar exams (Appendix E, Dec. of Stephen Castell PhD ¶35), and the fact that the software itself is provided by the Board, the Va. Board of Bar Examiners continues to enforce an antequated policy that has been in place since the early 1970’s that no applicant can obtain their essays. This policy is now being applied even to those who fall victim to a system software malfunction requiring hands-on technical assistance and system reboot. To this day applicants in Virginia are still being denied access to their essays, although 43 states and the District of Columbia afford their applicants such rights.

Petitioner has retained a leading computer forensic expert witness who has been able to conclude that "when a software glitch is encountered by an applicant, and should a dispute arise, the technically sound and reasonable way for the matter to be resolved is for there to be a policy in place for applicants to obtain their essays." Appendix E, Dec. of Stephen Castell, PhD, ¶2. If given the opportunity, he would also testify that the Board’s policy runs afoul of a core IT systems development principle and is a highly unusual way to administer any computer-based exam. Furthermore, because of Va. Code §54.1-108(1), applicants in Virginia who experience a system malfunction are prevented from the benefit of having the only workable remedy which was developed by the National Conference of Bar Examiners for resolving such disputes. Id. ¶38. Petitioner is advised that the technical assistance on standby at the exams are required to keep records detailing each incident of hands-on technical assistance and/or a system reboot provided at the exams.

Aside from the expert, Petitioner’s corroborative evidence includes witness testimony.

The first is a witness from New York who took the New York bar exam in 2007 and experienced a software system malfunction which required hands-on technical assistance and a reboot. Because he was able to obtain his essays under New York law, he could identify, unbeknownst to the New York board, portions of two of his essays that were cut short or erased. His essays were then regraded pursuant to the remedy developed by the National Conference of Bar Examiners, and he subsequently passed. The second witness is someone who can verify the magnitude of the problems experienced at the saving stage of the Virginia Bar Exam. The third witness is Petitioner himself, having fallen victim to the board’s software malfunctioning in both morning and afternoon sessions of the test requiring hands-on technical assistance and a reboot. He also observed others who had the same experience he did. Without a meaningful review mechanism in place, he lost a career position in a state’s attorney’s office as a result (Complaint ¶10, entitled "Parties & Standing").

Rather than hearing the case on its merits, the district court dismissed for lack of standing and lack of a triable Fourteenth Amendment due process question. Petitioner urges this Court to review because standing is satisfied by the injury to Plaintiff stated in Complaint ¶10, and the due process question is clear since the essays themselves represent sine qua non evidence in any software dispute with the board brought before the state supreme court.

There exists a universal right in every state for an applicant to bring a claim to the state high court under its inherent authority over the bar exam; nevertheless, without the essays, an applicant who experiences such a software malfunction is denied the liberty to state a claim upon which relief can be granted. Perhaps most importantly, precluding applicants under these circumstances from obtaining their essays also precludes the only available remedy as developed by the National Conference of Bar Examiners.

The National Conference of Bar Examiners, in response to a software system malfunction that affected New York’s July 2007 bar exam, developed an alternative grading methodology which first requires cooperation with the affected applicant. Where portions of the essays were identified as being truncated or lost, the following press release of the New York board describes how the method was applied:

"Fifteen of these candidates passed the examination based on their performance on the balance of the examination, with no credit being given for any missing essay. Seventeen
candidates failed the examination even when attributed a perfect score on any missing essays. The remaining 15 candidates were given estimated scores based upon their performance on the balance of the examination, and their probability of passing was computed."

Press release, New York Board of Law Examiners, Nov. 15, 2007, ¶5, available at http://web.archive.org/web/20071123015445/www.nybarexam.org/press.htm (last visited May 3, 2012).

Computer-based testing thus raises significant new issues that cannot be ignored and re-opens the question of what constitutes substantive and procedural due process protections following the bar exam.

REASONS FOR GRANTING THE WRIT

I. Unlike times past, there is today a compelling scientific explanation for revisiting the question of whether a bar examinee’s due process interest is infringed by a blanket rule that no applicant can obtain their essays following a computer-based bar exam.

Va. Code §54.1-108(1) specifically exempts examination papers of professional testing boards from the disclosure provisions of the Virginia Freedom of Information Act. See code section reprinted in Appendix F. The Board, in turn, follows an unwritten policy that no applicant can obtain their essays. This policy is applied uniformly to all, including those who undergo a software system malfunction in the Board’s testing software. The Fairfax Circuit Court addressed this issue in its order entered March 16, 2009 in Petitioner’s individual case, stating: "[the court] does not reach the issue of whether the Board has abused its discretion by adopting a policy to never release bar exam answers to applicants." Bolls v. Virginia Bd. of Bar Exmnrs., Case No. CL-2008-15212, n.1. Surprisingly, this important and timely issue has yet to be addressed on its merits in the context of a software malfunction by any state or federal court.

The Eastern District Court’s memorandum opinion acknowledges that applicants to the bar are to be afforded the protections of the due process and equal protection clauses of the Fourteenth Amendment. Appendix A, Opinion of the U.S. District Court, p. 7. However, it references Schware v. Bd. of Bar Exmnrs., 353 U.S. 232, a character and fitness case, to underlay an overly restrictive view on what protections fall within the scope of the Amendment. The court states: "federal courts have exercised restraint in reviewing non-discriminatory practices and procedures." Id. at p.8.

Instead of considering the novel issues raised by computer-based testing, the Court merely concludes that access to bar exam essays are traditionally not considered necessary for due process. Without addressing the evidentiary report of a leading computer forensics expert which pointedly states that this rule must be revisited (Appendix E, Dec. of Stephen Castell PhD ¶¶2, 35), the Court refers applicants who experience a software malfunction directly to the Virginia Supreme Court. Id. at p.8. But this rationale falls hopelessly short, because it fails to explain how it is possible to bring such a claim without the only evidence that could substantiate it, i.e. the essays. Petitioner’s own individual case demonstrates the point.

A. The Scientific Basis for the Argument

The Declaration of Stephen Castell PhD, included in the Appendix, identifies a clear Catch-22 that applicants are placed in when directed to the Virginia Supreme Court without their essays:

"I am informed that applicants have a legal right to petition the Virginia Supreme Court, which has the power to reverse the board’s assessments. I turn now to Mr. Bolls’ chart showing the various state boards’ policies [Appendix C] which places Virginia in a small minority of states that do not allow applicants rights to the essays. As of February 2009, when Mr. Bolls made the phone calls, 43 of the 50 states afforded these rights. As an expert witness in many high-profile cases both in American federal courts and the English High Court on matters relating to information and communication technology, software, systems, and services, I cannot conceive how it would be possible to bring a claim relating to a software malfunction without discovery of the item in question.  This leaves the question of what the means are of resolving such disputes."
Appendix E, Dec. of Stephen Castell PhD, ¶32.

The Board has already admitted that hands- on technical assistance and software system reboots are, in fact, occurring at every Virginia bar exam sitting. Nevertheless these applicants are prevented from protecting their interests because their essays remain cloaked in secrecy, becoming in a sense "unsolved mysteries." According to Dr. Castell, in ¶22 of his report, "[t]he need for a reboot in such circumstances is in my view a very real cause for concern."

Dr. Castell further alludes to an applicant who sat for New York’s computer-based bar exam in 2007 and who experienced a software system malfunction requiring hands-on technical assistance and a system reboot:

"Mr. Zeni’s personal account [Cmplt. ¶29] does show the confirmation of receipt of all essay responses by the board was proven to be inaccurate . . . Mr. Zeni obtained his essays as of right, and, through his assistance, he was able to identify for the board two separate essays where portions of his essays were missing, not just the one question where the crash occurred."
Appendix E, Dec. of Stephen Castell PhD, ¶26.

Mr. Zeni has also agreed to be a witness in this case. The impropriety of the state law and the Board’s corresponding policy is underscored further by Virginia’s departure from the normal way of operating a computer-based test:

"Virginia, as I understand it, does not allow applicants to obtain their essays. In my view and experience, this is an unusual way to design a system and immediately gives rise to concerns over validity and completeness checking, and difficulties in identifying and assessing any data corruptions or omissions potentially caused by system malfunction or otherwise. Clearly, to apply such a policy . . . is to overlook the possibilities of systemic error and to have no objective and even handed methodology of investigation in regard to ruling out such possibilities."
Id. at ¶30.

Perhaps most significantly, a remedy does exist but cannot be utilized in Virginia by the applicants due to Va. Code §54.1-108(1) and the board’s corresponding policy that no applicant may obtain their essays. The remedy was developed specifically in response to software system reboots experienced in New York’s July 2007 bar exam [Footnote:  ABA Journal.  Law News Now.  Available at http://www.abajournal.com/news/ny_bar_exam_marred_by_glitch/print/ ]which Dr. Castell concludes is consistent with what is being experienced in Virginia (Id. at ¶35). On this point Dr. Castell explains:

 "If, as in my view, there were in Virginia to be the reasonable, and technically sound, policy in place for applicants to obtain and inspect their essays, then, if such portions were thereby identified, it seems to me that there would then be an obvious and sensible remedy, viz, to employ the alternative grading methodology developed by the National Conference of Bar Examiners."

Id. ¶38.

Dr. Castell is able to conclude that when a software glitch is encountered by an applicant, and should a dispute arise, the "technically sound and reasonable way for the matter to be resolved is for there to be a policy in place for applicants to obtain their essays." Id. ¶2. The reason for that is quite simple, as Dr. Castell states:

"[T]here will inevitably be instances where cooperation with the applicant, the key creator of the critical data involved, is necessary, if not vital"
Id. ¶31.

The Eastern District Court avoids all of the above compelling reasons for why computer-based testing reopens the question as to whether access to the essays is an important protection to the applicant’s due process interest.


B. The overwhelming majority of States accord their applicants rights to the essays as the transition to computer-based testing is made.

Applicants to the bar in Virginia have a right, as do applicants in any state, to petition the state high court under its inherent authority with any grievance relating to the administration of the bar exam. With the advent of computer-based testing, a dispute that arises out of a software malfunction is certainly foreseeable and presents a valid and new kind of grievance. In such a situation, to withhold the essays is to withhold sine qua non evidence, which acts to completely vitiate the right to present one’s grievance to the state high court. Hence the Catch-22 . . . applicants are directed to the state supreme court but given no means to substantiate their claims. As the FifthCircuit pointed out in Parrish v. Board of Commissioners of the Alabama State Bar, 533 F.2d 942 (5th Cir. 1976), "access to bar examination papers was crucial to plaintiff’s case."

Petitioner’s argument is essentially for a modification or change to existing law based on changed circumstances. Before the software issues came into being, it was generally thought that the opportunity for reexamination provided an adequate means of exposing grading errors. Tyler v. Vickery, 517 F. 2d 1089 (5th Cir. 1975), cert. den., 426 U.S. 940 (1976); Whitfield v. Illinois Board of Law Examiners, 504 F.2d 474 (7th Cir. 1974). Thirty years after these decisions came down the computer-based test was introduced, raising a new question on what procedural safeguards are adequate to protect an individual’s rights should something go wrong with the software during the test.

Most states now offer a computer-based test, and most applicants now choose to take the test on the board’s special testing software. Appendix C, Chart, Statistics of the Computer Based Bar Essay Examination. The answer for the vast majority of the states is resoundingly clear: transparency. All except for seven states allow applicants access to their essays following the exam. Appendix C, Chart, Transparency Policies of State Law Examiner Offices (note that Colorado has recently joined the trend of allowing the applicants access).

There are a couple of state cases that demonstrate the point. The high court in Alaska called the board’s position a logical "hiatus" in that it required the applicant to demonstrate error without affording a device to locate the error. Application of Peterson, 459 P.2d 703, 709 (Alas. 1969). The high court of Arizona dismissed a petition for lack of sufficiency where it did not set forth "exact and complete particulars" of alleged unfair or improper grading of a particular set of exam papers. Application of Heaney, 476 P.2d 846, 848 (1970). For these reasons, any valid software dispute is bound to be dismissed without the only evidence to bear on the issue, i.e. the essays. With the onset of the computer age and high reliance placed on information technology, it is no longer proper to keep in place policies that withhold test essays. As Dr. Castell states, "there will inevitably be instances where cooperation with the applicant, the key creator of the critical data involved, is necessary, if not vital." Appendix E, Dec. of Stephen Castell PhD ¶31.


C. Applicants who experience a software malfunction have at a minimum a due process right to a responsible determination of the issues present.


A license to practice law is a well-recognized protected liberty interest. Not just the Equal Protection clause but the Due Process clause was specifically invoked by this Court as a limitation on state regulation of its bar. Schware v. Bd. of Bar Exmnrs., 353 U.S. 232, 238-239 (1957) ("A State cannot exclude a person from the practice of law or from any other occupation in a manner or for reasons that contravene the Due Process or Equal Protection Clauses of the Fourteenth Amendment") (emphasis added). See also this Court’s opinion in Greene v. McElroy, 360 U.S. 474, 496 (1959):

"[R]ight to follow one’s chosen profession comes within the 'liberty' and 'property' concepts of the provisions of the Fifth amendment to the Federal Constitution that no person shall be denied liberty or property without due process of law."

More recently in Goldberg, this court clarified the bare minimum of what constitutes due process. This Court stated:

"[T]he hearing can be tailored to the necessities of the situation and need not be a full scale, judicial trial of the issue, so long as it produces a responsible determination of the issues present."
Goldberg v. Kelly, 397 U.S. 254 (1970).

This is where the reasoning of the district court, and by extension the Fourth Circuit by summary affirmance, falls short. Without considering the content of the expert report (reprinted in the appendix) and the compelling reasons it gives for why applicants must be able to obtain their essays, it simply directs applicants who experience a software malfunction to the Virginia Supreme Court. But the opinion fails to address ¶¶ 9, 30, and 45 of the Complaint which speak to how the Virginia Supreme Court is not allowing the essays to be disclosed following a software problem, and legitimate complaints are being wrongfully dismissed as a result. In Petitioner’s own individual case previously, for example, he filed a sworn affidavit detailing the software malfunction that affected both himself and others. Nevertheless, the court would not overrule the Board’s policy that no applicant can obtain their essays. The Board advises applicants with a software dispute that all results are final and there is no appeal. There is no built-in review process, whether formal or informal, whereby the software dispute could even be properly investigated. The Board, as an agency of the state supreme court, is also specifically exempt from the Va. Administrative Procedure Act.

This is wholly unreasonable and unfair in a computer-based test. When dealing with bar admissions, the Virginia Supreme Court and the Board are essentially acting as a single administrative agency. Rogers v. Supreme Court of Virginia, 772 F.2d 900 (4th Cir. 1985) (P.4). Since the actions of the Virginia Supreme Court and the Board directly impact the right to practice one’s chosen profession, they are subject to the constitutional limitations of the 14th Amendment and federal judicial review.


II. To deny standing for applicants that experience software malfunctions to challenge a state law which prevents access to their essays poses an impenetrable barrier to judicial scrutiny of legislative action.


The district court concludes that there is no Article III standing in this case, finding no harm to Petitioner and no personal stake in the matter. Appendix A, Opinion of the U.S. District Court, p. 6. This, however, directly contradicts the allegations of the Complaint, viz ¶10, stating as follows:

"To date his essays have been held in secret even though his case has been described by a notable computer forensics expert as a 'textbook case' where cooperation with the applicant is 'necessary, if not vital.' His essays were withheld from him, and his case never could be heard on its merits. He was denied his license and lost a job in public service."

The circuits that have addressed the issue are in agreement that aside from fraud and coercion, probative facts that point to manifest unfairness and other serious grounds or circumstances warrant federal court review of state bar examiner policies. 7 Am. Jur. 2d Attorneys at Law §20. See Chaney v. State Bar of California, 386 F.2d 962, 967 (9th Cir.1967); Feldman v. State Bd. of Bar Exmnrs., 438 F.2d 699, 704 (8th Cir. 1971) ("manifest unfairness on the part of the bar examiners are grounds for inquiry into the integrity of bar examination results"); Whitfield v. Illinois Bd. of Law Exmnrs., 504 F.2d 474, 478 (7th Cir. 1974); Cf. Scinto v. Stamm, 620 A. 2d 99, 103 (1993) ("evidence that the grading system is not effective in revealing grading errors or that it inadequately guards against the risk, if any, of an erroneous deprivation of an applicant’s interest"). Given the recent transition to computer-based testing in Virginia (2005), it is especially important for federal courts to ensure that the applicant’s rights are being upheld at the state level.


A. Contrary to this Court’s holding in D.C. Court of Appeals v. Feldman, 460 U.S. 462, the lower court seeks to avoid jurisdiction over individual rights questions of national importance.

This Court in Feldman made a fine line distinction between what cases can and cannot be brought by an aggrieved bar applicant in federal district court. There being a circuit split on the matter at the time, the Court adopted the rule in Doe v. Pringle, 550 F.2d 596, 597 (10th Cir. 1976) that federal courts cannot review particular denials of bar admission but can review general attacks on the constitutionality of a bar rule. See Feldman, 103 S. Ct. at 1317:

"United States district courts, therefore have subject-matter jurisdiction over general challenges to state bar rules, promulgated by state courts in nonjudicial proceedings, which do not require review of a final state-court judgment in a particular case."

The instant case represents such a case requiring federal court jurisdiction. It involves a facial challenge under the Due Process Clause to Va. Code §54.1-108(1) which prevents applicants to the bar from obtaining their essays. Upon this law the Board acknowledges it has formed the basis for its blanket rule that no applicant can obtain their essays.

Since its inception, the computer-based bar exam has had incidences throughout the country where essays have been erased or cut short so as to negatively impact the grading assessments. See, e.g., the remedy employed by the National Conference of Bar Examiners in cooperation with the New York Board of Law Examiners following New York’s July 2007 bar exam. "Candidates whose essays had been corrupted and irretrievable were either awarded full credit for those essays, or those questions were thrown out and the candidates were simply graded on the balance of the examination." Appendix E, Dec. of Stephen Castell, PhD ¶33. Law examining boards of Kentucky and New Jersey have also admitted to experiencing problems similar to this.

States such as Virginia appear to be conveniently ignoring the issue. By not allowing applicants to obtain their essays, they maintain an illusion of perfection, even though, as Dr. Castell states, "Without an allowance for applicants to obtain their essays, in my opinion the board may very well have a serious problem on its hands and not even know it." Id. ¶30. It is thus paramount that this Court uphold the Due Process clause to vindicate individual rights at the state level, where information technology raises significant new issues not accounted for in the previous case law.


B. The question of what constitutes due process in the context of a software malfunction is ripe for review.

The overwhelming trend of the States is to allow the applicants to access their essays. Appendix C, Chart, Transparency Policies of State Law Examiner Offices. Only seven (7) states do not allow such access, although they administer a computer-based test, most of which have over 50% of their applicants relying on the satisfactory performance of the board’s chosen software. Appendix D, Chart, Statistics of the Computer Based Bar Essay Examination.

What all the States have in common is an unqualified right accorded to the applicants to petition its state’s high court with any grievance relating to the bar exam. The issue is thus squarely presented before this Court whether the introduction of computer-administered testing throughout the country is cause for a change in policy in those states that do not allow their applicants rights to the essays, especially when they experience a software malfunction.

　
C. Proper characterization of injury-in-fact

The injuries to Petitioner and a significant number of others that Petitioner complains of are not, as the district court would have it, merely software system malfunctions. Though these malfunctions are a seminal reason for a change in law and policy, the actual injury to Petitioner and others similar situated is quite different, much more insidious, and ongoing. In Virginia a bar examinee has a right following the release of the results to bring a complaint to the Virginia Supreme Court, which has the power to reverse exam assessments under its inherent authority. Woodard v. Va. Bd. of Bar Exmnrs., 454 F. Supp. 4, 5 (E.D. Va), aff’d 598 F.2d 1345 (4th Cir. 1979).  But this rightis fatally handicapped when there exists an unwritten policy followed by the Board that no applicant can obtain their essays, which policy is rooted in Va. Code §54.1-108(1) denying Freedom of Information Act requests for the same.

The injury that Petitioner complains of is therefore an unreasonable hindrance to his and others’ access to the Virginia Supreme Court, namely the right to present necessary evidence in support of a complaint before that court.

The evidence in this case would prove, by expert and eyewitness testimony, that the Board is engaging in a pattern of improper activity by applying the policy in question uniformly to applicants who experience a software system malfunction as described in Complaint ¶28 and Petitioner’s sworn affidavit. This policy of uniform application has been in existence since the 1970’s, long before the software system was introduced to all applicants in Virginia in 2005. This is going on while there exists a law that traces at least as far back as 1919 in Virginia (see Va. Code §54.1-3929, reprinted in Appendix F) that requires specifically the Board to preserve an individual’s essays, presumably for any kind of dispute that should arise, which would now include software disputes.

In sum, Petitioner does in fact have standing. He was directly affected by the software system malfunction in 2008, and he has firsthand experience with the law in question being applied to him just as he sought to substantiate a valid claim before the Virginia Supreme Court. As it turned out, it was impossible for the Virginia Supreme Court to hear the merits of his case because evidence crucial to the software dispute, the essays themselves, were unfairly held in secret.


D. Violation of access to the courts has been found with significantly less substantial impediments to an individual’s rights than what is seen here.

Bar examinees, regardless of which state they are from, continue to have rights following the release of the bar examination results. "[T]he due process clause requires the state to employ fair procedures in processing applications for admission to the bar and, therefore, that an applicant who has failed the bar exam is entitled to some procedural protections." Whitfield v. Illinois Board of Bar Examiners, 504 F.2d 474, 477 (1974).

Procedural protections fail, as in the case of Virginia, when there exists an impediment which is significantly difficult for an individual to surmount. "In order to work a violation of the right to access the courts, it is not necessary for a statute to produce a procedural hurdle which is absolutely impossible to surmount, only one which is significantly difficult." C.J.S. Constitutional Law §2156 (2009). How much more warranted is judicial review when the obstacle poses an impossible barrier. Without the essays, in any software dispute, it is utterly impossible to resolve the dispute.


E. The courts have a duty to open, rather than close, the door to the courthouse where a denial of standing would pose an impenetrable barrier to any judicial scrutiny of legislative action.

Va. Code §54.1-108(1) is a product of the Virginia General Assembly, and it directly impacts the rights of every applicant to the Virginia Bar. Where "a denial of standing would pose, in effect, an impenetrable barrier to any judicial scrutiny of legislative action, the court’s duty is to open, rather than close, the door to the courthouse." C.J.S. Actions §112 (2009). If an applicant such as Petitioner does not have standing to challenge the law, then that leaves open the question of who would have such standing. Petitioner experienced a software crash, then he sought to obtain his essays, was precluded from obtaining them so that he could petition the Virginia Supreme Court,and remains subject to the same flawed law (and corresponding policy) should he take the exam again in his home state. Petitioner’s software malfunctioned at the saving stage of both morning and afternoon sessions of the test, requiring hands-on technical assistance which was to no avail, and finally a system reboot, all of which is described by a prominent computer forensics expert as a "textbook case" of a software malfunction and "very real cause for concern." Appendix E, Declaration of Stephen Castell PhD ¶31 and ¶32. Further, Respondent has conceded that the software reboots are, in fact, occurring at every exam.

There is nothing paradoxical about a bar exam applicant who experienced the system reboots described in ¶28 of the Complaint by himself and a significant number of others to eventually seek to change an improper policy that denied him and others meaningful review. The Supreme Court of Virginia is the proper forum for these disputes but it is utterly impossible for a litigant to bring a software related dispute before the Virginia Supreme Court if he cannot first investigate, make an informed decision, and particularize the evidence in the complaint. Essentially, the Board is usurping the authority of the Virginia Supreme Court by unilaterally deciding which claims can and cannot proceed against itself. This case therefore goes to the very core of the integrity of the grading process and a sense of fairness in the American judicial system.

　
F. One or more of the essay answers for 47 candidates to New York’s July 2007 Bar Exam could not be recovered following software symptoms consistent with that of Virginia.

Software malfunctions have led to data loss, and consequently misgrading, in other state bar examinations. See, e.g., a press release by the New York Board of Law Examiners stating that one or more of the essay answers for 47 applicants could not be recovered following "technical problems" experienced with the software. Press release, New York Board of Law Examiners, Nov. 15, 2007, ¶5, available at http://web.archive.org/web/20071123015445/www.nybarexam.org/press.htm (last visited May 3, 2012). Eric Zeni, witness in the case at bar, can testify as to the press release’s authenticity. The ABA Journal reported the incident with more specificity as "problems saving and uploading [the] essays." ABA Journal, Law News Now, July 26, 2007, available at http://www.abajournal.com/news/ny_bar_exam_marred_by_glitch/print/ (last visited May 3, 2012).

After reviewing a statement provided by Eric Zeni, Dr. Castell was able to conclude that the software symptoms observed in Virginia are consistent with those of New York. Appendix E, Dec. of Stephen Castell PhD ¶35. However, unlike in New York, applicants in Virginia cannot avail themselves of the remedy developed by the National Conference of Bar Examiners for when a software malfunction happens. See press release, supra, ¶5. That is because applicants are prevented from obtaining their essays pursuant to Va. Code §54.1- 108 (1).

It is also important to note that Eric Zeni can testify that the board did not identify the portions of two of his answers that went missing as a result of the software malfunction. He did.

Those applicants in New York who experienced a software malfunction, and who were then able to obtain their essays and apply the above remedy, would not have had the same opportunity to protect their license had they taken the test in Virginia. That should be a compelling enough reason for any one of the applicants in Virginia who experienced software symptoms consistent with that of New York to have standing in a federal court to challenge the very law that prevented their cases from being heard.


III. Under the factors set forth in Matthews v. Eldridge, 424 U.S. 319, post examination administrative procedures of seven (7) States are not constitutionally sufficient given the transition to computer-based testing.

The seven States that preclude their applicants from obtaining or reviewing their bar exam essays are *Virginia, South Carolina, Georgia, South Dakota, Hawaii, Arkansas, and Missouri. Appendix C, Chart, Transparency of State Law Examiner Offices.  [*Footnote:  Colorado has recently switched to allow such access during the pendency of this litigation].Every one of these States now administers a computer-based test.  Appendix D, Chart, Statistics of the Computer-Based Bar Essay Examination.   [*Footnote:  Hawaii is the most recent addition, having recently introduced the computer-based test during the pendency of this litigation.  However, Hawaii has kept its policy in place that no applicant can obtain their essays]. 

The Fourth Circuit has explained that since the state supreme court acts as a court of first impression in the unique case of the bar exam, and since the Board is an agency of the supreme court, the court is "acting as an administrative agency, rather than as a court of appeals" and it is "appropriate to determine whether or not [ ] allegations concerning the review procedures of the court and the Board make out a viable claim." Rogers v. Supreme Court of Virginia, 772 F.2d 900 (4th Cir. 1985) (P4).

Petitioner thus urges this Court to grant review and consider the factors outlined in Matthews below.


A. Factors in analyzing judicial or administrative procedure
Petitioner contends that in states that do not allow applicants to obtain or review their essays following the bar exam, the post examination right to petition the state’s high court is vitiated in any software related dispute. Resolution of the issue of whether administrative procedures provided here are constitutionally sufficient requires analysis of the governmental and private interests that are affected. In Matthews v. Eldridge, 424 U.S. 319, 96 S. Ct. 893 (1976) this Court considered three factors when analyzing whether an individual is constitutionally entitled to a particular judicial or administrative procedure.

　
1. Private interest affected by the action.
Many applicants, like Petitioner, have decided to use their law degree to go into public service. Some, like Petitioner, have secured postgraduate fellowships at these organizations where they would be taken on full time once they pass the bar. Without a timely review process in place, applicants whose results may have been affected are effectively penalized for six months. If they work for a public service organization they will in all likelihood lose their job.

2. Risk of erroneous deprivation of such interest, through the procedures used, and the probable value, if any, of additional or substitute procedural safeguards.
As it stands, it is an open question as to how a valid dispute relating to the testing software can even be presented without discovery of the item in question, i.e. the essays. According to Dr. Castell, expert witness in this case, "This leaves the question of what the means are of resolving such disputes." Appendix E, Declaration of Stephen Castell PhD, ¶32. Unbeknownst to them, this is a risk now undertaken by over 50% of the applicants in the seven states that do not allow access to the essays. Appendix C, Charts, Transparency Policies of the States, Statistics of the Computer Based Test.

Essay answers have been known to be partially or totally missing following software symptoms consistent with those that are occurring at the saving stage of the Virginia Bar Exam. Witness Eric Zeni and expert witness Stephen Castell can provide corroborative testimony on this point. Additionally, the Engineers’ Notes of standby technicians who assist the Virginia Bar Exam will also confirm that system reboots are occurring at the saving stage of every exam. The significance of this is that such symptoms lead to grading inaccuracies.

The value of releasing essays to applicants who take the computer-based test is in how it safeguards the integrity of the bar exam. First, it promotes the appearance of a fair test. Second, it encourages graders to identify and isolate essay responses that appear to have been corrupted. If they do not, the applicants will, as Eric Zeni can attest. Grading any of these responses leads to inaccuracy, which ironically defeats the underlying purpose behind having a proficiency test. Dr. Castell is also prepared to testify that a rule precluding access to essays following a computer-based test (CBT) violates a core IT systems development principle that in any CBT there are times when cooperation with the applicant is "necessary if not vital." Id. ¶31. Only then can a remedy developed by the National Conference of Bar Examiners be used.

3. The Government’s interest, including the function involved and the fiscal and administrative burdens that the additional or substitute procedural requirements would entail.

Neither the Virginia Supreme Court nor its Board has been able to identify an interest or harm that would occur if an applicant could obtain his/her essays. As demonstrated in other jurisdictions, releasing essays is a safeguard that would not add one penny to the cost, which is typically borne by the applicant for a small fee.  [*Footnote:  For example, the Florida bar examiners charge $50 and the Alaska bar examiners charge $10 for the service]. 

The Board may decide to adopt an informal process involving cooperation with the applicants who experience a software system malfunction. Technicians at the test site, working on behalf of the board, already know who required hands-on technical assistance and which computers required a system reboot (which they recommend). For example, in North Carolina, a state that allows applicants to obtain their essays, the board suggests that an applicant write to the chair about significant flaws in the grading.

Without employing any such measure, a right to obtain essay answers would add nothing to the cost. In each of these seven so-called "non transparent" states, applicants already do have a right to petition the state’s high court. These state courts can dismiss cases, and regularly do with relative ease, according to their discretion. However, if an applicant had the essays in hand that showed the impact of a software malfunction, as Eric Zeni would have been able to show, then the court would be remiss in dismissing such a case. 

                                CONCLUSION

The district court’s dismissal of this case runs contrary to this Court’s decision in Feldman, 103 S. Ct. at 1317 that general challenges can be brought to bar examiner policies and procedures. This Court may intervene in cases involving states, particularly "in those situations where it feels the error is so serious as to constitute a fundamental unfairness in the proceedings." [*Footnote:  R. Stern & E. Gressman, Supreme Court Practice 6.27, p. 460 (5th ed. 1978)]. 

The recent national transition to computer-based testing for state bar exams has been accompanied by a near uniform change in policy toward allowing applicants rights to their essays.  For those seven states that do not, there is a legitimate question as to whether due process is being violated.  Inaccessibility of the essays following a software system malfunction precludes the possibility of obtaining a fair hearing on the merits consonant with the requirements for a responsible determination of the issues present.
 
As this Court in Schware put it, bar examiners are subject to both the Due Process and Equal Protection clauses in the conduct of their duties. Schware, 353 U.S. at 238-239. A careful reading of the district court’s opinion on p.8, summarily affirmed by the Fourth Circuit, uncovers a reluctance to deal with due process cases that really should not be there. Even if Due Process cases are more rarely presented, it is all the more important that when they are they do not go ignored simply because more cases involve the Equal Protection clause. Certainly this Court would not stand for the proposition that the Virginia Supreme Court may employ procedures in flagrant violation of the Fourteenth Amendment Due Process clause without any kind of federal judicial review.

The issue is ripe for review as there is no precedent for what constitutes procedural due process protections in the context of a software malfunction. As this Court has stated, " ‘Due Process’ is, perhaps the least frozen concept of our law- the least confined to history and most absorptive of powerful social standards of a progressive society." Griffin v. Illinois, 351 U.S. 12 (1956). Va. Code §54.1-108(1) was enacted into law at a time when the General Assembly could not have foreseen the vast implications of computer software performance on society as a whole, let alone on individual rights as we see today. In the past, such laws preventing access to the essays have generally been upheld; however, the advent of computer-based testing demands that courts take a second look. The evidence shows that applicants in other parts of the country who have been able to obtain their essays have indeed proven that the same software symptoms observed at the Virginia Bar Exam have led to corruption of their essays, and misgrading as a result. Furthermore, a workable remedy does exist that requires cooperation with the applicants.

Petitioner’s personal stake in the outcome should have been readily apparent to the court below. The injury to his due process rights is still ongoing as the evidence necessary for a proper review of his case continues to be held in secret pursuant to an improper policy following a computer-administered exam. He furthermore has a vested interest in exposing a flaw in a bar admissions process that he has a standing right to undergo again in order to practice his chosen profession, in his home state.

Petitioner respectfully requests this Court to grant review.


Respectfully Submitted,

Jonathan Bolls
Attorney Pro-Se