VIRGINIA BAR EXAM: LACK OF TRANSPARENCY

Update 36.  Last month I provided the deans of Virginia's law schools with the twenty-page expert report in relation to my case and all candidates who experience similar technical problems saving their bar exam essays using the Exam4 software.  Also included were my sworn statement as to what I experienced at the test site (and observations of others) and print-offs of the policies of neighboring state board of bar examiners who, unlike Virginia, recognize the due process rights of the applicants and allow access to the graded essays following the exam.  See, for example, Maryland's Rule 19-207, available at http://www.courts.state.md.us/ble/pdfs/baradmissionrules.pdf

Last year, when the Courts Committee held a hearing on this, the Board told them that it has decided to change its unwritten policy of not allowing anyone access to their essays to allowing a review of the essays after second and subsequent failures, but still not after the first exam.  This is unacceptable and poses no remedy to applicants like Eric Zeni in New York, who was told his essays were properly received only to discover that was in fact not the case once he actually obtained his essays as of right. Mr. Zeni also provided a well written letter to the Courts Committee in Richmond explaining how critical the right to obtain his essays in New York was to identifying and correcting a grading mistake as a result of a technical failure of the board's software (his symptoms were very similar to what we are experiencing in Virginia). 

With the help of friends, an analyst for a federal intelligence agency and a lawyer for the Federal Reserve, I have prepared the following letter presenting the issues to the deans of the law schools of Virginia.  It is time for Virginia to join the rest of the States in ensuring transparency in the system and guaranteeing the due process rights of the applicants.


December 3, 2016

Re:  Transparency of the Virginia Bar Exam

1.  Virginia does not allow an unsuccessful candidate to obtain their essay answers or review them until a second failure, even in situations where there is evidence of a system software malfunction or other technical failure in the grading.  This undercuts the spirit and purpose behind Virginia Code Section 54.1-3929, which requires the Board to preserve the essay answers for one year.

2.  This is fundamentally unfair, makes little sense, and is contrary to the recommendations of professional associations, the approach taken by 43 other states, and the analysis of outside experts.

3.  This policy seriously adversely impacted myself and others, and will continue to do so.  It reflects poorly on the Commonwealth, calls into question unnecessarily the credibility of the exam results, and can create the impression that Virginia law schools are lagging in successfully preparing their students to pass the bar exam.

4.  It is high time for a change in policy now that the bar exam has made the transition to being administered as a computer-based test.


Dear [Dean]:

As Justice Brandeis once said, "Sunlight is the best disinfectant."  I am writing to you and the other Virginia law schools in reference to a matter that directly affects the integrity of the bar admissions process in Virginia, a problem that directly impacts law school rankings and has gone largely ignored by the courts and the legislature after it was presented to them.  And that is the existence of an unwritten rule that no applicant can access their bar exam essays after their first bar exam.  This rule was applied to me after I experienced a system software malfunction and reboot on Exam4 software provided by the Board of Bar Examiners (see my sworn affidavit enclosed and, separately, article published in the Computer Law & Security Review by myself and a leading computer forensics expert).  As you will see when you read the article, these software problems are occurring nationwide and are directly tied to misgraded bar exams (fn.  for example, see this story of a New York bar applicant who experienced a software problem and how he was able to protect his interests by obtaining his essays as of right:  http://www.newyorkpersonalinjuryattorneyblog.com/2008/02/i-passed-the-new-york-bar-exam.html).  I have no doubt that my essays were impacted, as the essay score was nowhere near how I had actually performed.  Shortly after the exam I was able to reconstruct my essays by memory and was able to compare them to the approved solution.  The relatively recent national transition to computer-based testing is therefore a significant reason for a policy change at this time for the Board to allow applicants to obtain their essays upon request, especially if something went wrong at the test site (fn.  this would be an inexpensive proposition, as the other states are simply charging a nominal fee to cover the costs of making the copies.  Florida, for example, charges $50; North Carolina $20; New York $40).   43 of the states have made this transition already (see enclosed chart) given the inherent risks of computer-based testing, and Virginia would do well to follow suit in order to protect the due process rights of the applicant.

I brought a constitutional challenge to the policy of withholding the essays from review in federal district court in Richmond and filed a 20-page expert report in support (see report enclosed).  This report was filed pro bono by Dr. Stephen Castell, a leading computer forensics expert with over forty years of experience as an IT consultant in some of the highest profile cases in the world.  For example, in 2012 he led a team of three experts in  Australia's T-Card highway transportation system failure (ITSL & ERG v. PTTC, a $200M claim against the government for contract termination was ultimately settled on the eve of trial largely due to Dr. Castell's involvement and findings).  See http://www.smh.com.au/it-pro/government-it/tcard-battle-settled-out-of-court-20120217-1tegm.html.  He also testifies in high dollar patent cases in American federal courts for law firms such as Jones Day and Morrison & Foerster.  Virginia is certainly very fortunate to have him looking at this issue (cv attached).  Dr. Castell describes how my case was a "textbook case" of a system software failure and how there is no way to properly determine completeness checking without including the applicant in the process.  In Par. 20 of the expert report he states as follows:  "In no way does this pose a remedy to, or even a sensible investigation of, any potential corruption problem caused by the Exam4 software itself . . . for which Mr. Bolls could not have been responsible."  I have consulted two additional experts who have arrived at the same conclusion as Dr. Castell, which may be one reason why the Board has never brought forward an expert witness of its own.  In the end, the case was dismissed for lack of subject matter jurisdiction, but one good aspect that came out of it was this:  in their Answer, the Board admits to having made a uniform and unwritten rule that no applicants may have access to their essays and that rule was applied to the circumstances of my case.  I have enclosed a copy of my sworn affidavit on what took place at the test site for your review as well, which clearly affected many others as well as myself.  To this day, my essays remain cloaked in secrecy, encrypted on my laptop, and I have never seen them to verify completeness (fn.  it is also interesting to note that Virginia Code Section 54.1-3929 requires the Board to preserve an applicant's essays for one year.  The Board, however, has interpreted discretion accorded to it in another code section as license to make a uniform rule that no applicant can access their essays.  This, of course, is not a proper use of discretion). 

Meanwhile I took my MBE (multiple choice) score, which was high enough to waive into other jurisdictions, and waived that portion of the test into Washington, D.C.,  taking the remainder of the essay exam only.  I successfully passed in D.C. on my first attempt, in a test where over half of the applicants failed.  I then took the Maryland Bar Exam and passed on the first attempt as well, scoring highly.  I have been working as a full-time magistrate in Fairfax, VA.  I also served as president of the Washington Inn of Court in D.C. last year and I teach trial advocacy at a local prep school.  So fortunately things here worked out for me despite the actions of the Virginia Board of Bar Examiners (fn.  a member of the admissions committee of the Maryland Board of Law Examiners actually apologized to me for what the Virginia Board did and said they hope that I will have a better experience with the legal community in Maryland).  Nevertheless, I am cognizant of the fact that the facts of my case militate so strongly in favor of a change in policy at this time to allow applicants access to their essays in a computer-based test (which are preserved for a year by Va. Code Section 54.1-3929) that it would be unfair to future applicants for me to look the other way on this issue.  As previously stated, I was one of many applicants who experienced the software system malfunction and presumably all of them were treated the same way I was if they had later asked to see or obtain their essay answers.

To borrow the words of the computer forensics expert:

"Virginia, as I understand it, does not allow applicants to obtain their essays.  In my view and experience, this is an unusual way to design a system and immediately gives rise to concerns over validity and completeness checking, and difficulties in identifying and assessing any data corruptions or omissions potentially caused by system malfunction or otherwise.  Clearly, to apply such a policy, even to the circumstances of Mr. Bolls' case, is to overlook the possibilities of systemic error and to have no objective and evenhanded methodology of investigation in regard to ruling out such possibilities.  Mr. Bolls knows what he wrote and, with his input, it may be discovered that software malfunction was the cause of the discrepancy.  Without an allowance for applicants such as Mr. Bolls to obtain their essays, in my opinion the board may very well have a serious problem on its hands and not even know it."   Par. 30 of expert report.

The Board saw all of this information, including my sworn statement as well as the expert report; nevertheless, they still refused to release the test essays, even though they had the discretion to do so.  When asked by senior Circuit Court Judge Alden in Fairfax why not, they told the judge in open court that they used their discretion to make a blanket rule that no applicant can access their essays.  This is why when it was ultimately determined that the Board was exempt from the Administrative Process Act, in her dismissal the judge made it clear in her written opinion that she is preserving the issue of whether the Board abused its discretion in making a blanket rule that no one can review their essays.  As it stands, I was not able to view my essays following the system reboot at the test site and I have not been allowed to see them ever since.  Furthermore, due to my blog which tells this story (and has nearly 11,000 hits from the legal community), I have been informed by applicants as recent as last year that these software problem persist at the test site for many people.   Presumably, all of them are being offered the same sham remedy that I was (see below, an offer I did not accept as this would lead to a misleading result).  The Board's lack of cooperation is likely a result of not wanting to employ the National Conference of Bar Examiners' alternative grading methodology for when a software malfunction occurs.  See Par. 5 of http://web.archive.org/web/20071123015445/www.nybarexam.org/press.htm.  This would necessarily include all applicants affected, which they consider an inconvenience to their usual way of doing things prior to the advent of computer-based testing.

SHAM REMEDY OFFERED BY THE BOARD

The sham remedy being offered to applicants who experience a software system crash while the essays are being saved is as follows:  compare the encrypted file on their Exam4 software (saved on the applicant's computer) to what was graded.  But herein lies the problem:  the file saved on the Exam4 program on the applicant's computer itself is likely corrupted to begin with.  Therefore, what the graders graded was necessarily a duplicate of such potentially corrupted data.

As Dr. Castell states in his report:

"It is my opinion that the Board's offer to compare Mr. Bolls' encrypted file against the one graded does not in any way address the matter at hand.  That is to say, an exact match between the two does not inform the board or the Supreme Court of Virginia that there was no data corruption in this instance.  It merely confirms that the work is that of Mr. Bolls, without providing any illumination as to whether or not the work itself had been corrupted.  The only way that I can see this can be reasonably resolved is for there to be a policy in place for applicants to obtain their essays."
Par. 36 of the Expert Report

"Like Mr. Zeni, applicants such as Mr. Bolls would and in my view should then have the opportunity to identify places where portions of the essays went missing or were corrupted in some way.  If, as in my view, there were in Virginia to be the reasonable, and technically sound, policy in place for applicants to obtain and inspect their essays, then, if such portions were thereby identified, it seems to me that there would then be an obvious and sensible remedy, viz to employ the alternative grading methodology developed by the National Conference of Bar Examiners." 
Par. 38 of the Expert Report

What ends up happening is most applicants, generally unfamiliar with computer data storage methods, will take the Board up on their offer to do this comparison exercise, and be naively satisfied with it.  In reality, had they actually seen their essays, it is likely they would be seeing what the applicants in New York observed in 2007- missing or altered data that impacted the grading.  This is what Mr. Zeni experienced after facing a similar reboot of his software in the middle of the bar exam in New York.  See the enclosed letter from Mr. Zeni to the Courts Committee in Richmond.

Because of my experiences with the Board, I no longer have confidence in the integrity of bar admissions in Virginia, so rather than retaking the exam I have decided to go to other jurisdictions to become barred.  However, I have recorded every stage of my legal case as well as a hearing before the Courts Committee on my blog.  Included in the packet for your review are the proposal and other materials set before the committee.  Ultimately, the committee concluded that my case, while badly handled by the Board, was not enough to take action that would ensure applicants rights to review the essays.  However, it is important to remember that I was certainly not the only one affected (and many more continue to be affected), and our test results remain unsolved mysteries in that first exam sitting.

Since the litigation of this case, the Board has changed its policy to allow applicants to review their essays on the second or subsequent failure; however, the first bar exam is the most important.  This is the exam where many applicants (like myself) have jobs on the line; law school rankings are also impacted by the first exam results.  It is essential that the first bar exam be graded accurately and any software failures include a satisfactory review which according to my expert (and all other experts that I have consulted) necessarily includes the applicant in that process.  It is no longer proper for a computer-based test to have a rule that no applicant can review their essays afterwards, and it is high time for a change in policy.

The American Bar Association, the National Conference of Bar Examiners, and the American Association of American Law Schools have drafted a "Comprehensive Guide to Bar Admission Requirements 2016."  See http://ncbex.org/publications/bar-admissions-guide (page X).  In Section 25 it has this to say:

"25.  Rights of Failing Applicants . . . [a]n applicant who fails a bar examination should have the right, within a reasonable period of time after the announcement of the results of the examination, to see the applicant's answers to the essay questions and the grades assigned thereto, and to compare each of these answers with an approved answer." 

I have included in this packet the printed rules of surrounding jurisdictions Maryland, District of Columbia, North Carolina, and New York that adhere to this best practice.  These are but a small sample of the vast majority of the States that have the same transparency protections in place for the applicants (see chart, enclosed). 

If you are interested in this subject and in using your influence to help Virginia join the 43 other states who have already protected the due process rights of the applicants, I would be happy to discuss or answer any questions you have of me.  What needs to be done to fix this problem is what most of the other states area already doing- to allow applicants to obtain, not just review, their essays.  And this must be after every examination, not just second or subsequent failures (see legislative proposal, enclosed).  We live in a time where there is such a high degree of reliance on computer systems, which has many benefits, but we must ensure that our policies are appropriately tailored to the changed circumstances to ensure that rights are protected while the Board of Bar Examiners performs its functions.  Indeed, that was the clear intent of the General Assembly when they enacted Va. Code Section 54.1-3929, which requires the Board to preserve essay answers for one year.

Thank you for your attention to this matter and I look forward to hearing from you.


Regards,

Jonathan Bolls
Magistrate, 19th Judicial District
Fairfax, VA

Cc:  the law schools of Virginia

SEE LETTER BELOW

Update 35.  On February 1, 2016 this matter was brought to the attention of the Civil Courts Subcommittee by Delegate Vivian Watts of northern Virginia.  After looking at the facts of my case, she agreed that the time has come to bring transparency to the Virginia Board of Bar Examiners, especially now that it is operating a computer-based test.  In Richmond, she brought forward House Bill 550, which is to ensure that all applicants have access to their essays as originally intended by the Legislature.  I was also called as a witness to the system software failure that took place affecting me and many dozens of others at the test site who were then told we could not see what impact there was on our essay answers.  Present at the hearing were the committee members, led by Chairman Greg Habeeb of Southwest Virginia as well as an audience of Virginia residents.  Delegate Habeeb is very public about the need for transparency and accountability in government.

Delegate Watts spoke first, explaining that the bill is in line with what 43 of the States are already doing and offers much needed transparency and accountability in the grading process.

I explained to the committee that I serve as a magistrate and have since passed the bar in multiple jurisdictions, on the first attempt, and scored highly.  I am and have always been certain that the essay score I received for the Virginia Bar Exam was inaccurate and highly likely a result of the system software malfunction I and numerous others experienced at the test site.

In my statement I first laid out a blow-by-blow account of what happened at the bar exam with the Board’s software- how it froze during the saving stage of the essays and how a technician went to work hands-on but was unable to fix the problem and finally, as a last resort, instructed me to reboot the system (turn it off and on again) and then save to a USB thumb drive without looking at the essays.  One of the delegates blurted out, “You should have filed for an injunction!”  I later went into my legal case and so he was able to find out that is exactly what I did do. 

I then provided each member a chart of all the States that do allow access as well as a personal letter [see letter below] addressed to the committee and written by an applicant in another jurisdiction who experienced the same thing we did here in Virginia.  The difference was that he was able to obtain his essays by right.  When he got them, he discovered portions were missing, and indeed, one answer response was completely erased.  His answers were graded as if those were his complete answers.  Since 48 others who obtained their essays after his test had similar observations of erased material, this forced a corrective process in the state of New York not previously in existence.  Importantly, prior to the applicants obtaining their essays they were reassured by the Board that they need not be concerned and that their complete answers were received.  No so.

The delegates were additionally provided with a 20-page expert report submitted in support of my case.  Dr. Stephen Castell, a leading computer forensics expert who has testified in high dollar cases for companies such as Motorola, Citibank, and Harvard Securities PLC, on both the Plaintiff and Defense side, was able to form a conclusion that, given what happened at the test site (and still does in the most recent exams from what I am told by multiple sources), my concerns were well justified.  Further, he says there is no way for the Board to properly investigate without including the applicants, the actual writers of the essays, in the review.  As he says, “they [the Board] may have a very serious problem on their hands and not even know it.”  In other words, all of the tests of those affected remain unsolved mysteries. 

Just last year ExamSoft Worldwide, Inc., the sister software company to the one that Virginia uses (Extegrity) was sued in a class action by the applicants for software failures at a test site in another jurisdiction, resulting in a multi-million dollar settlement.  See http://lawschoolcafe.org/2015/05/20/examsoft-settlement.  Case No. 1:14-cv-22950.  Protecting the software company may be one reason for the lack of cooperation with Virginia bar applicants who are falling prey to system software malfunctions and who notice a significant scoring discrepancy in how they believed they performed on the test.  

Finally, I shared some research with the committee I conducted on the history of Va. Code Section 54.1-3929, which since 1919 has required the Board to preserve the essays for one year.  Sure enough, I discovered that prior to 1973 applicants in Virginia were, in fact, able to have access to their essays, which was the fundamental reasoning behind requiring the Board to preserve them for a year.  Sometime after 1978 the language pertaining to the right of access was removed, leaving an absurd result of the Board being still required to preserve the essays but the applicants left unable to access them, even in the most compelling circumstances.  This paradox lays the groundwork for the ongoing violation of Due Process in the Commonwealth of Virginia for anyone who has a legitimate dispute with the Board.  The very evidence that is needed, i.e. the essays, is being withheld in order to prevent a fair hearing.

I left them with this: even if the right to access the essays was for the sole purpose of allowing applicants to be able to prepare and improve the next time, is that not good enough reason alone to get rid of the blanket rule that no applicant can access their essays?  

 

The committee seemed a bit lackadaisical unfortunately and not as focused as one would like on the important issues at hand.  Chairman Habeeb seemed to actually be the only proactive one.  He first said, as the matter was being called forward, that this ultimately has to do with whether we trust the Virginia Board of Bar Examiners.  Some did take an interest when I explained the procedural history of my legal case and how venue was changed from the Fairfax Circuit Court to the Virginia Supreme Court due to the fact that the Board managed to have itself exempted from the Administrative Process Act (which reviews for agency wrongdoing).  And the Virginia Supreme Court has no original jurisdiction and dismisses all of these cases as a matter of course.  This is why, as I explained, the Board behaves this way towards people as opposed to other professional licensing boards, such as the Architects, which deal directly with public safety.  There is no doubt that if a candidate for an architect license experienced a software glitch in the computer-based test, and their board refused to cooperate with that applicant, a hearing could be heard in Fairfax Circuit Court.  Should it be any surprise then that the Board of Bar Examiners, as the only such organization shielded from judicial review, would treat people so poorly? 

 

The new Secretary of the Board, Catherine Hill, spoke against the bill.  She cited to a lot of statistics about the pass rates, etc. which really were beside the point for the most part.  She decided not to address any of the computer forensics evidence and simply argued for the status quo, which she argued was working just fine.  When asked by Chairman Habeeb if she has looked into this problem, she said yes and said vaguely that there was no problem found.  There should have been a follow-up question as to how did you look into it and what technical expertise was brought to bear in any supposed review of the situation.  Instead, in response to the actual follow-up question of whether there have been any statistical anomalies over the years, she simply said no. 

Besides these softball questions, there unfortunately wasn’t much in the way of critical analysis.  Any semblance of rigorous debate to get at the truth of this was wholly lacking.  Either the committee was ill equipped to handle a matter of this complexity or there was a simple deference to a government agency (many of them are licensed by the Board). 

Whatever the case may be, the committee tabled the issue from any further consideration this year.  At the end, Chairman Habeeb told the Board (which showed up as a team) they “really do not want us legislating this so make sure that people are happy.”  Folks in the audience could be heard in agreement with that statement.

 

Judging by one of his comments, Delegate Habeeb revealed some of his rationale by saying that we do not want to make bad law based on a badly mishandled case.  While that may be true in certain instances, that is certainly not the case here for two reasons.  First, I was by far not the only one affected by the system software failure; and second, these problems are persisting up to the most recent exams I am told by some who follow this blog.  The truth of the matter is that badly mishandled cases will continue unless a good law is made now.  It has worked just fine to require transparency in almost all of the other States. 

 

But, as we joked at the hearing, we all know that Virginia is usually the last to change.  Here was Virginia’s opportunity to do something right for the People, but passed up the opportunity.      

 

 

LETTER FROM ERIC ZENI, ESQ TO VIRGINIA GENERAL ASSEMBLY

January 30, 2016

Courts Committee
General Assembly Building
201 N. 9^th Street
Richmond, VA  23219

 

Re:  Virginia House Bill 550

 

Dear Members of the Courts Committee:

 

I would like to submit this statement in support of House Bill 550, which would allow bar exam applicants access to their essays.  I am an attorney admitted to practice in New York State.  I sat for the July 2007 administration of the New York Bar Exam, which I registered to take using a computer.  I believe that my experience with the New York Bar Exam shows why it is important to make the applicant’s answers available to them.

 

When I sat for the bar exam, there were wide-spread problems with the exam software that was used.  In my testing location, a great many people were affected by the software glitches.  Later we learned that these problems occurred state-wide.

 

The exam software froze twice while I was taking the exam.  As I was typing my first essay, the software froze and a technician came to assist.  The technician restarted my laptop, which I did not think was a good idea, but eventually got the software running again.  I then completed the first essay and moved on.  As I was half way through the second essay, the software crashed again.  A technician came and advised me that he could take my laptop to a table in the back of the exam room and work on it.  I glanced back to see a table in the rear corner of the room with computer technicians, equipment, and anxious test-takers- a veritable computer triage area.

 

At that point, I told the technician to give me an answer booklet.  I completed the remainder of the second essay and finished the exam on paper, the old-fashioned way.  At the end of the day, we were instructed to transfer our files on a thumb drive and upload them.  At the conclusion of the test, I spoke with the technicians and the head proctor who advised me that the software had a “backup system” and that no data would be lost.  I believe that I even received an email confirmation from the software company that they had received my test responses.

 

Ultimately, when the exam results were released, I was told I failed by four points.  At that time, in New York, test-takers who failed were allowed to request copies of their answers.  After I received the copies of my essays, I discovered that my response to the first essay written on the computer was incomplete and that the first half of my response to the second essay (which I had typed on my computer) was completely missing.

 

Although New York does not allow appeals of the test results, I contacted the Board of Law Examiners and explained that significant portions of my responses were missing and explained the software malfunctions that had occurred.  Several weeks later, the Board of Law Examiners informed me that they had done some sort of “review” and were able to locate the missing portions of my essays.  My essays were re-graded and I was informed that I did, in fact, pass the bar exam.

 

It is important to note that if New York did not allow an applicant who failed to request and review their essays, I would never have discovered (or been able to prove) that the graders did not have my complete answers and that the software malfunction had resulted in the loss of significant portions of my essay responses.  The ability to review my answers was truly life-changing for me- it meant that I did not have to sit for the bar exam again and I was able to be sworn in on admission day with my peers and begin my career.

 

This is why I write in support of H.B. 550, which I believe will protect the interests of the applicants.  I believe that it is only fair that applicants have access to their responses, especially if they need to prepare to retake the examination.

 

If I can be of assistance to your committee, please feel free to contact me if you have any questions.

 

Sincerely,

S/

Eric Anthony Zeni

VIRGINIA BAR EXAM: LACK OF TRANSPARENCY

Update 34.  My letter to the Virginia General Assembly has been received favorably and a timeline has been requested of me as well as the name of the chief justice of the Va. Supreme Court at the time the case was presented, Leroy Hassell.  Below is the procedural history of the legal case I brought on behalf of myself as well as all applicants for the Virginia Bar:

DECEMBER 2008    FAIRFAX CIRCUIT COURT.   Between the end of October and beginning of November I attempted to obtain my essays from the Virginia Board of Bar Examiners.  I was told repeatedly that once the results are released they are final and there is no possibility of review.  At the end of November I filed an emergency motion in Fairfax Circuit Court seeking a mandamus, a judicial command order, under the Administrative Process Act, to compel the release of the essays.  Senior Judge Leslie Alden heard the arguments for a half hour.  An assistant attorney general, who came up from Richmond, did not at that time dispute that there was a software failure.  When asked by the judge, "what remedy is available to aggrieved bar examinees," she stated:  "that is an interesting question."  The judge then took the case under advisement and requested I file a supplemental brief on the applicability of the Administrative Process Act (see enclosed).  It was discovered that the Virginia Board of Bar Examiners is among a list of exempted entities from the general application of the APA (by the Code of Virginia, as one of three agencies of the Virginia Supreme Court).  Unlike practically all other state agencies, therefore, the Fairfax Circuit Court had no jurisdiction.

Note that in her decision (enclosed), Judge Alden found that while there is no explicit statutory authority to review the decisions of the Board, the Virginia Supreme Court retains such inherent authority and that I should therefore invoke the original jurisdiction of the Virginia Supreme Court in this case.  Note also that the judge, in footnote 2, states:  "the Court does not reach the issue of whether the Board has abused its discretion by adopting a policy that it can never release bar exam answers to applicants."

In the opinion the court suggests:  "To the extent the Petitioner seeks a mandamus ordering the Board to release his bar exam answers, the Petitioner should invoke the original jurisdiction of the Supreme Court.  Va. Const. art. VI, Section 1." 


MAY 2009 VIRGINIA SUPREME COURT (Chief Justice Leroy Hassell).  As instructed by the lower court, I brought the mandamus action to the Virginia Supreme Court under Va. Const. art. VI, Section 1.  Along with the complaint, I included my sworn affidavit as to the details of the software failure that affected my test as well as observations I made about other applicants who experienced the same difficulties saving the test essays.  I also disclosed how 24 applicants were found to have had essays mispositioned in the software system from the morning session as evidenced by an announcement made at the test site.  In addition, in the complaint I referenced the case of an applicant in New York who experienced the same software symptoms that I did who was able to prove misgrading of his test essays only once he was able to obtain his essays. 

The court simply dismissed the petition altogether.  What little the court did say was mandamus does not lie to compel a discretionary act (which it does, in the case of abuse of discretion, which was clearly a part of the claim).  In a petition for rehearing, I claimed that there was no other forum to hear the case and by not hearing the case (or such cases) the court is propounding a due process violation of the Federal Constitution.  Still, the court denied rehearing, again without explanation.


AUGUST 2010   U.S. DISTRICT COURT FOR THE EASTERN DISTRICT OF VIRGINIA.  Judge Robert Payne.  Because the Virginia Supreme Court declined to exercise its own inherent authority over the bar exam, I was essentially left without redress in the Commonwealth of Virginia.  I then brought the case to federal district court as a violation of due process on the part of the Virginia Supreme Court and its Board of Bar Examiners.  The case precedent for jurisdiction here is Rogers v. Supreme Court of Virginia, 772 F. 2d 900 (4th Cir. 1985) ("we think that, when that court (Virginia Supreme Court) reviews or declines to review a decision of the bar examiners as to an individual examinee's status, the court is acting as an administrative agency, rather than as a court of appeals.  Therefore, we find it appropriate to determine whether or not Rogers' allegations concerning the review procedures of the court and the board make out a viable claim.").  See also Richardson v. McFadden, 540 F.2d 744, 750 (4th Cir. 1976) ("It is beyond question that the bar examiners are subject to the requirements of due process and equal protection in the conduct of their duties") and Whitfield v. Illinois Board of Bar Exmnrs., 504 F.2d 474, 478 (1977) ("the due process clause requires the State to employ fair procedures in processing applications for the admission to the bar and, therefore, that an applicant who has failed the bar exam is entitled to some procedural protections"). 

I obtained a leading computer forensic expert who submitted a brief to the court where he was able to already form certain conclusions which corroborated that my concerns were indeed justified and that the descriptions of what occurred in New York's software failure matched what is being observed in Virginia.  See attached expert report of Stephen Castell PhD.  The expert also concluded that the National Conference of Bar Examiners has already developed an "obvious and sensible remedy" to use for situations that come up with the computer based test (see last sentence of Stephen Castell's report).  It also characterizes my case, in Paragraph 31 of the expert report, as a "textbook case" and that the Board's unwritten policy of nondisclosure "is an unusual way to design a system and immediately gives rise to concerns over validity and completeness checking."

In the complaint, Paragraph 17, I allege that the unwritten policy of nondisclosure of test essays has the effect of rendering the right of every bar exam applicant to petition the Supreme Court of Virginia defective and illusory.  That is because of the basic requirement that one "plead with particularity" so as to make out a cognizable claim.  Without such particularity, cases are dismissed by the Virginia Supreme Court (and all courts) on a routine basis.  See Application of Peterson, 459 P.2d 703 (Alaska 1969), where the Alaska Supreme Court called the board of bar examiners' position a "logical hiatus" in that it required a bar examinee to demonstrate error without affording him a device in which to locate that error.  The essays are known as "sine qua non" evidence in this instance- evidence without which there exists no case.

In my prayer for relief, I ask for no monetary compensation whatsoever.  I merely ask that the unwritten policy of nondisclosure be struck down as unconstitutional and that my essays be released to me at once. 

Defendant's Answer.  The Board's Answer to the Complaint (enclosed) admits to the unwritten policy of nondisclosure.  See Paragraph 13.  It furthermore confirms that it has been applied in this case and has been applied uniformly to all applicants.  In chambers, the assistant attorney general was corrected by Judge Payne when she misstated that it has always been this way, because he said it was not when he took the exam.

Court's Decision (see enclosed).  Without addressing the merits, the court dismissed the case on the premise that I was challenging the Board of Bar Examiners' unwritten policy with respect to my own exam as opposed to all applicants.  In fact, I was doing both.  As the prayer for relief states first and foremost:  "That this Court take the necessary steps to protect the rights of all future applicants to the Virginia Bar by declaring Policy of Nondisclosure and Policy of Finality unconstitutional."

Court of Appeals for the Fourth Circuit (see enclosed). The decision of the lower court is sustained without giving any reasons.  I have enclosed my brief.  The attorney general's office did not bother to respond, other than with a paragraph or so.


JUNE 2011 U.S. DISTRICT COURT FOR THE EASTERN DISTRICT OF VIRGINIA.  Judge Henry Hudson.  Since the previous case was dismissed for challenging the policy of nondisclosure with respect to my own exam, this time I brought forth a new action simply challenging the policy on its face as unconstitutional and seeking no relief for myself whatsoever.  See the enclosed complaint.  By this point, I had waived in my multiple choice score from Virginia (not the essay portion) into D.C. and took only the essay test in D.C.  D.C. runs a fully transparent system, and with a 60% failure rate, I still was able to pass on my first attempt.  Virginia is a mere 15% fail rate for first time takers.

Board Threatens Sanctions.  When this second case was filed, which becomes a part of the record, the attorney general's office claimed that I was bringing "vexatious lawsuits" and sought sanctions against my D.C. license.  Previously, this threat was made as an attempt to get me to withdraw the case.  And my response letter (see response to the sanctions motion, attached) was sent to the assistant attorney general as well as personal and confidential to Attorney General Ken Cuccinelli, explaining my reasons why this case had to go forward.  Even though this assertion as to "vexatiousness" was completely without basis, this issue was litigated and the court found in my favor and denied the motion.  Therefore, the case was able to proceed. 

Court's Decision.  The court issued a 9-page opinion.  At the heart of its rationale, found on the bottom of p. 7, is the idea that federal courts will not review due process or equal protection cases unless they involve cases of discrimination.  In other words, racial discrimination cases are the only ones they will review.  Other cases cited to in my brief that show federal review is not and cannot be just limited to discriminatory cases were simply ignored and not addressed in the opinion.

Ironically, as opposed to the prior district court decision which concluded it had no jurisdiction because I was applying the facts too much to my own case, this district court concluded no jurisdiction on exactly the opposite ground:  "In crafting his complaint, Plaintiff seeks relief on behalf of all future bar applicants and appears to distance himself from the group of individuals directly affected by the statute."  (p. 6 of the opinion).  Ignoring my primary contention that a policy withholding essays in a disputed essay exam prevents a case from going forward to the Virginia Supreme Court, the court lackadaisically states that applicants can petition the Virginia Supreme Court or take the bar exam again.  Without transparency, and given my personal experience with the Board, it should be quite obvious why I would never take the Virginia bar exam again without an appropriate change in policy.  This forced me to go elsewhere to gain admission to the Bar.

Court of Appeals for the Fourth Circuit (enclosed).  Sustained the lower court's decision, without explanation.


PETITION FOR CERTIORARI BEFORE THE U.S. SUPREME COURT (enclosed booklet).  Denied October 1, 2012.  The Supreme Court is careful to note that a denial of cert is by no means an expression on the merits of the case.  All legal remedies at the state and federal level have therefore been exhausted.  Despite the evidence and witness testimony available, the case never was heard on its merits.   

VIRGINIA BAR EXAM: LACK OF TRANSPARENCY

Update 33.  The courts, having punted this issue over to the General Assembly, have demonstrated the flaw in the procedural protections afforded to applicants to the Virginia Bar.  With the enclosed letter (see below) I have notified a select group of state delegates and senators who may be willing to sponsor legislation that will bring Virginia up to speed with the other states in terms of transparency.  The Board of Bar Examiners is on record in federal court as admitting to having an unwritten rule, unbeknownst to the candidates, that prevents any applicant from obtaining their essay answers, a rule they admit was applied in this case.  I am informed by a witness to the latest exam in 2014 that the Board's software failures persist and continue to affect significant numbers of applicants at every succeeding exam.  This is what happens when people, including judges, look the other way.

I have since taken the D.C. and Maryland bar exams and passed on the first attempt.  Both systems operate transparently.  I have decided not to take the Virginia Bar Exam again because I have lost confidence in the integrity of the system.


Dear Delegate/Senator:

I am writing to you about a proposal that I have enclosed which would remedy a fault that exists in the current procedures of the Virginia Board of Bar Examiners.  Let me first say that the Board is an administrative agency with both judicial and delegated legislative powers.  And while there is support at the local level in northern Virginia for this proposal, this will require your office's support to succeed because the Board of Bar Examiners has refused to cooperate in making this adjustment towards transparency.  I am an attorney and a magistrate in Fairfax and, based on what I have observed and experienced, there is no doubt that this proposal is what is necessary to protect the applicants and correct an ongoing problem with the new computer based bar exam.  I attempted to litigate this issue, but the Virginia state courts and federal courts have decided that it is essentially a political question for the General Assembly and decided not to hear the merits of the case.

SOFTWARE FAILURES AT THE VIRGINIA BAR EXAM

Virginia's bar exam is now being administered on computer software provided to the applicants by the Board of Bar Examiners.  The Board contracts with an out-of-state software company to provide this software so they can administer their test for over a thousand applicants every year.  When I took the Virginia Bar Exam, I was one of a significant number of applicants who experienced a software system failure that occurred at the moment when the essay answers are to be saved and submitted.  Members of the Board's technical team attempted to work on the problem but ultimately said the system had to be rebooted (abruptly turned off and then turned back on).  One of the female applicants was in tears when this happened to her.  An announcement was made that over the lunch break they had already discovered that 24 applicants had answers misplaced in the system from the morning session.  This was exactly what happened in New York the year before:  see attached, New York Law Journal "Software Snafus Upset Test Takers," July 26, 2007.  I was then forced, without being able to see and verify that the answers were intact, to transfer the potentially corrupted data onto a USB thumb drive and hand it in to the graders.  Upon receiving my essay score months later, I noticed a significant scoring discrepancy between the reported score and how I had performed on that day.  No other information was disclosed.  Of course, I immediately requested to obtain my essays.  In response to my request, however, I was told there existed no corrective process and they had an unwritten rule, applied uniformly, that no applicant can obtain their essays for any reason.  Even under these circumstances, where I explained that I had a job lined up at the prosecutor's office and produced a sworn affidavit saying exactly what had occurred at the test site, the secretary of the board refused to cooperate.  Moreover, I was able to obtain a 20-page expert report from a leading computer forensics expert who concluded that the fault lay not with my computer but with the board's software system.  As a result, I was given no redress and consequently lost a career position at the Arlington Commonwealth Attorney's Office.

SHAM REMEDY

The only remedy offered to me was a sham remedy wherein the board agreed to examine only what was on the USB thumb drive and do a comparison with what was printed off from the USB drive for the graders to grade (recall that the data was transferred from the software to the USB drive only after the software system failure . . . at no time was I permitted to confirm if the answers were still intact).  The computer expert, and three other experts I interviewed, have been unequivocal in their conclusions that this is no remedy at all and does not address the underlying issue, namely whether the data that was saved on the USB drive in the first place was corrupted.  Therefore, a comparison exercise like this accomplishes nothing because a copy of already corrupted data will necessarily be a match; nevertheless, it is still corrupted data.  This is misleading.  Therefore, there is no proper way to go about this other than to allow the applicant to obtain his/her essays and determine if what was graded includes everything that was written.  That is what the National Conference of Bar Examiners recommends for the computer-based test, and that is how it was handled in New York.  All of this was explained to a federal judge but, as I mentioned, the judge avoided the question, ruling this was essentially a political question for Virginia's legislature.

SAME PROBLEMS CONTINUE TO THE PRESENT DAY

This occurred in 2008, and I have a witness who confirms that he observed that the board's software failures continue to affect the applicants up to the most recent examination in 2014.  Furthermore, this is a problem that is happening nationwide since the advent of the computer based test in 2005.  For example, in New York's 2007 computer-based examination, it is well documented that as many as 47 applicants had essays that were either missing or cut short by the software after experiencing the same symptoms that I and others experienced in Virginia.  See enclosed Press Release, New York Board of Law Examiners (Nov. 15, 2007) (available on archive at http://web.archive.org/web/20071123015445/www.nybarexam.org/press.htm) (specifically paragraph 5 which reads "one or more of the essay answers for 47 candidates could not be recovered").  These applicants were allowed to obtain their essays; and once they did, it forced a corrective process that was not previously in existence.  In the case of Eric Zeni, this made the difference between passing and failing.  See enclosed Eric Zeni's account in the New York Personal Injury Law Blog.  Had Eric Zeni been in Virginia when this happened, there would have been absolutely no recourse.  This cannot stand, and it violates the due process rights of the applicants who expend an enormous amount of time and resources into this test and have secured positions contingent on their passing. 

LEGISLATIVE HISTORY

Allow me to provide you some background on the legislative history of this issue.  In 1919 the General Assembly enacted Va. Code Section 54.1-3929 which requires the Board of Bar Examiners to preserve an individual's examination papers for a period no less than one year following an examination.  This provision was relied upon by applicants and their attorneys whenever a dispute would arise.  This important protection to the rights of the applicant and the integrity of the system as a whole has been effectively nullified sometime in the early 1970's when the Board put into effect an unwritten policy, unbeknownst to the candidates, that prevents applicants from being able to obtain or review their essays.  This policy is being applied uniformly without exception, a fact I have the Board as having admitted to on the record in federal court.  Prior to that, applicants could obtain their essays.  This is a fact that I know because U.S. District Court Judge Robert Payne in Richmond replied that he could have gotten his answers when the Board attempted to say it has always been this way.  Since 2005 the Board has been administering a computer-based bar exam for the essays, a condition that demands a change in policy at this time.

NATIONAL MOVEMENT TOWARDS TRANSPARENCY

In light of the national transition to computer-based testing, 44 of the states now afford their applicants an opportunity to review or obtain their essays, for any reason.  See enclosed chart, "Transparency Policies of State Law Examination Offices."  I hope that you will see that it is time for Virginia to acknowledge the new realities of computer-based testing and the need for transparency in the system.

The vast majority of the other state boards of bar examiners have rules that protect their applicants.  In Virginia, the General Assembly sets laws from which the Board must derive its rules.  Va. Code Section 54.1-3929 mentioned above is an example of one such law.  The only way for this problem to be solved is for the General Assembly to act, because the Board is insistent on resisting this change.

ABOUT ME

To provide you with a brief background on myself, I went to William & Mary Law School (Class of 2008) as well as the College of William & Mary after transferring from the U.S. Military Academy at West Point.  I have since taken the bar exam in the District of Columbia and Maryland, having passed both on the first attempt and have been admitted to practice in the federal courts and pending admission in the U.S. Supreme Court.  However, I have not taken the Virginia Bar Exam again because, with an unwritten policy like this one in place, I have lost confidence in the integrity of the system.  For more background on my story, I have published an article along with the expert I consulted in the Computer Law & Security Review (2013) (attached). 

PROPOSAL

Please see my attached proposal, which includes language taken from other states which Virginia can look to for guidance to solve this problem.  The Virginia Board of Bar Examiners is a state administrative agency with both judicial and delegated legislative powers. 

This nonpartisan issue affects not just candidates for the Virginia Bar but by extension it affects the people of Virginia, causing well qualified lawyers trained in Virginia to leave the Commonwealth and seek admission elsewhere.  Most legislation relating to the Virginia Bar Exam will be sought by the Board, but this one is for protecting the applicants from the Board's iniquitous unwritten policy of nondisclosure of test essays.  I will certainly make myself available to provide a statement or sworn testimony before any committees as necessary.  Furthermore, the expert report and his contact information is available upon request.

Thank you for your time, and I look forward to working with you on this important matter affecting the Commonwealth. 

Regards,

Jonathan Bolls

Virginia Bar Exam: Lack of Transparency

Update (32):  The following was just published in this quarter's edition of Computer Law & Security Review (Volume 29, Issue 4 (2013), p. 446-449, Computer Based Testing, available at www.sciencedirect.com :

"Computer based testing - Bolls v. W. Scott Street, Sec'y of Va. Bd. of Bar Exmnrs., August 11, 2009, case no. 090915"

By

Jonathan Bolls and Stephen Castell PhD

ABSTRACT

The most significant change to law licensing in the United States is the recent reliance on computer-based testing for the bar exam in all of the fifty states.  Virginia, for example, has been providing the computer-based test on a wide scale since 2005.  New Jersey was the very first, in 2001.  Since then, almost all of the states are now administering a computer-based test.  The following comment piece is meant to inform the computer and technology law community how the use of testing software must necessarily be accompanied by additional procedural protections and corrective processes that may not have existed previously.  Chief among these is the availability of discovery of the item in question, namely in this case the essays in a computer-based bar exam, which constitutes evidence that is essential in vindicating one's rights when a software failure occurs.

ARTICLE

      In July of 2008 I (Jonathan Bolls) took the essay portion of the Virginia Bar Exam on special testing software provided by the bar examiners.  At the conclusion of the morning session, the software froze while in the process of saving the essays.  There was a team of technicians on standby and the candidates were instructed to seek their assistance if anything went wrong with the software.  At the conclusion of the test, when the proctor asked if there were any problems, a significant number of hands went up including my own, visibly overwhelming the full team of technicians on standby.  It took a full 15 min before one could come around to assist me.  He worked on my computer hands-on and, when that didn't work, he instructed me to reboot and try saving the answers again.  So I did.  But after the reboot I was not able to see any of the content of what supposedly was being saved.  Then I had no choice but to transfer the file to a USB drive which was then handed in for grading.  That afternoon, prior to the start of the second session, the candidates were informed by loudspeaker that over the lunch hour they discovered that 24 candidates had essays which were "misplaced" in the USB drives.  These candidates did not know who they were and would receive further instructions in their afternoon test booklets on how to rearrange their essays properly.  After the afternoon session, I again experienced the same software malfunction that happened in the morning requiring hands-on technical assistance, which ultimately failed, and had to reboot.  Again, I was by far not the only one.

     Months later I received my results and became aware of a major scoring discrepancy in my essay score.  With a career position hanging in the balance, I immediately contacted the Board of Bar Examiners and requested that copies of my essays be sent to me.  My request was denied at once by the secretary of the board simply because, as he stated, there is no review or appeal process available and the results are final.  Furthermore, there exists an unwritten rule that no applicant can obtain their essays.

     I began to do some research and it was not long before I discovered that software system malfunctions are happening throughout the different state bar exams of the country and, in fact, data has been found missing, misplaced, or otherwise altered in some way.  Indeed, it has been proven that misgrading happened as a result.  Most interestingly, perhaps, is the fact that these affected applicants had been previously reassured (just as I was) that their answers were intact.  It was not until they received their essays as of right (which the vast majority of the states allow) where applicants became so outraged that they forced a corrective process not previously in existence.

     With a career position at a district attorney's office on the line, I had no choice but to bring a mandamus action (judicial command order) in Fairfax, Virginia to force the state agency to release the essays.  At the hearing, the local judge asked the attorney representing the state what process is available to aggrieved bar applicants, to which the answer was "that is an interesting question, Your Honor."  This set the case on a constitutional detour that I would never have expected to be on so soon out of law school.

     The case was taken under advisement, and while I was informed by a judicial law clerk that the judge was persuaded that the essays had to be released, it was ultimately determined that since the Board is an arm of the Virginia Supreme Court, only that court could issue such an order.

     Before taking the case to the Virginia Supreme Court, I conducted some research of my own, and what I discovered was interesting and troubling.  The year before (2007), applicants who sat for the New York bar exam complained of software glitches and system reboots in the process of saving the test essays, basically the same symptoms that I and numerous others in Virginia experienced.  See New York Law Journal, available at http://www.abajournal.com/news/ny_bar_exam_marred_by_glitch/print/   One key difference between New York and Virginia, however, is that applicants in New York have a right to obtain their essays.

     This right proved crucial for 47 applicants who discovered portions of their essays went missing, were duplicated, or otherwise altered in some way by the software system failure.  Initially, it was the testing board's position that there was no cause for concern, that all the essays were received in their original and complete form.  Even after the board released a press release about the 47 (New York Bd. of Law Exmnrs, available at (5th Paragraph Down) http://web.archive.org/web/20071123015445/www.nybarexam.org/press.htm ), we know of at least one applicant in particular (Eric Zeni) who demonstrated that the 47 was not all that were affected (New York Personal Injury Law Blog, available at http://www.newyorkpersonalinjuryattorneyblog.com/2008/02/i-passed-the-new-york-bar-exam.html ). 

     Eric Zeni, for example, demonstrated that two essays, not just one, were cut short by the software.  He has told me privately that the board actually tried to place the blame on him, as if it was somehow his fault their software wasn't working properly.  In the end, he was able to prove two of his essays were impacted by the software failure and should not have been graded because they were not reflective of his performance on the test.  After a proper review, his failing grade was therefore changed to a passing grade.  New York is therefore an example of how transparency upheld the integrity of the system- despite some level of inconvenience to the bar examiners themselves.

     Since 2007, the National Conference of Bar Examiners has created a remedy in response to what happened in New York, which includes identifying the essay portions that were affected and then grading the balance of the examination.  See New York Board of Law Examiners' press release, Par. 5, available at http://web.archive.org/web/20071123015445/www.nybarexam.org/press.htm

     Prior to bringing the case to the Virginia Supreme Court, I called every state board of bar examiners' office and inquired whether applicants have rights to the essays following an exam.  42 of the 50 states said they do (with Colorado most recently changing its policy to allow this right, increasing the number to 43).  Virginia stood as one of a small minority to operate a computer-based test with an unwritten rule that no applicant can obtain their essays, even in the event of a software system malfunction.  The other states that are still doing this include Arkansas, Georgia, Hawaii, Missouri, South Carolina, South Dakota, all of which are currently administering computer-based tests.

     One right that all applicants in each of the States have in common is they may present their grievance to their state supreme court, which has inherent authority over bar admissions.  As in the case when any legal claim is brought, evidence is needed to show injury.  In this case, only the release of the essays could do that.  The Board merely cited to an unwritten rule instituting in the early 1970's that no applicant can obtain their essays.

     I therefore had no choice but to bring a mandamus action, seeking a command order of the state supreme court to compel a public official to do his duty under an abuse of discretion claim.  I have confirmed that in my case no guidelines for discretion were followed.  No investigation of the software failure took place whatsoever.  It simply came down to a blanket rule that no applicant can obtain their essays.  With my sworn affidavit, Eric Zeni's testimony for what happened in New York just the year prior, the existence of a remedy developed by the National Conference of Bar Examiners, and the presence of a witness who can confirm my observations at the test site, the court still refused to hear the case or order the release of the test essays.  That being the highest court in the state, I was left without recourse and without being able to have my day in court.

     Because of the failure of Virginia's judicial system to provide any forum for handling software claims of bar examinees, I decided to bring a constitutional challenge in federal court to the Virginia state board's policy preventing applicants from obtaining their essays even following a system software failure.  Such a rule runs directly afoul of the due process protections of the applicants, particularly given the recent changed circumstances of the computer-based test.  The relief I was requesting was simply the release of test essays.  No request for damages or monetary compensation was included in state or federal court as I was seeking merely to vindicate my own rights and fix the obvious problem for all future examinees.

     First, I needed a computer forensics expert to look at this issue and provide the science behind the common sense as to why this rule is no longer appropriate in the environment of computer-based testing.  I had the great fortune to come across Dr. Stephen Castell, an IT Consultant of the Year Professional Award Medalist of the British Computer Society (http://www.CastellConsulting.com). Over the course of his thirty-five year career, Dr. Castell has become a leading international expert in his field and has testified in the English High Court, the Sydney Commercial Court, and American Federal Courts, for both plaintiff and defense, and for many major clients worldwide.  He has acted in both the longest (GEC-Marconi v LFCDA, 1991-1992) and largest (AirTours v EDS, 2001; total claims $250M pounds sterling) computer software development contract lawsuits heard in the English High Court.  After hearing about this case, he was kind enough to submit a 20-page report pro bono in support of the case. 

     The federal court had all the evidence that the Supreme Court of Virginia had with the added benefit of Dr. Castell's report in which he was able to form certain preliminary conclusions and opinions based on what happened at the test site.

     First, he was able to conclude that there was no way for a proper review to take place without including the applicant, the key creator of the data.  That is because, as he states, the need for a reboot under these circumstances is "a very real cause for concern," and "the nature of computer-based testing systems is that there will inevitably be instances where cooperation with the applicant, the key creator of the critical data involved, is necessary, if not vital . . . [i]t is as important also to remember that Mr. Bolls was not the only one affected, and his case may prove helpful to improving the accuracy, integrity, and reliability of systems if, like Eric Zeni, corruption of the essay responses is discovered." 

     Secondly, he was able to conclude that the Board's offer to compare what was saved on the USB drive for hand-in to the graders with what was saved in encrypted format on my computer was just a sham remedy because the software system failure took place before the data was even transferred to the USB drive.  Therefore, as he states, "[T]his can only merely confirm that the file was written by Mr. Bolls using the Exam4 software.  In no way does this pose a remedy to, or even a sensible investigation of, any potential corruption problem caused by the Exam4 software itself, or through other (temporary or permanent) system fault, for which Mr. Bolls could not have been responsible." 

     Third, he affirmed the usefulness of the remedy as developed by the National Conference of Bar Examiners, which is entirely foreclosed to applicants in Virginia until there exists a means for obtaining the essays:  "If, as in my view, there were in Virginia to be the reasonable, and technically sound, policy in place for applicants to obtain and inspect the essays, then, if such portions were thereby identified, it seems to me that there would then be an obvious and sensible remedy, viz, to employ the alternative grading methodology developed by the National Conference of Bar Examiners."  It is important to note that no expert was brought forward by the Board of Bar Examiners; in fact, I had found two other experts who came to the same conclusions that Dr. Castell did. 

     The federal court nevertheless dismissed the case without a hearing or determination of the merits on the grounds that I was challenging the agency's rule as applied to my own exam and not as a rule applied to all.  This actually wasn't true as I made it very clear in my complaint to the court that besides relief for myself (in the form of releasing the test essays) I was also challenging the rule on its face and sought in my relief to have the rule struck down for future applicants who will have to undergo a computer-based test ("That this Court take the necessary steps to protect the rights of all future applicants to the Virginia Bar by declaring Policy of Nondisclosure and Policy of Finality unconstitutional").  The appellate court affirmed without explanation.

     While preparing for the next step in the case, I decided to take the bar exam in another jurisdiction, and was able to successfully pass the Washington, D.C. bar on my first attempt (handwritten not typed).  I have taken a job as a magistrate judge in Fairfax, Virginia, a position that is appointed by the Supreme Court of Virginia.

     It was becoming clear something was very wrong with how the judicial system in Virginia was approaching software disputes with the Board of Bar Examiners.  Either they did not want to take the time to understand the science and hear the expert or they were simply hostile to the issue.  There is no doubt that with the evidence presented, this case should never have been dismissed without a proper hearing. 

     Taking the cue from the last decision, I brought the case again in federal court, this time not asking for relief for me at all, but rather a declaratory judgment that the rule that no applicant can obtain their essays and the corresponding state law preventing Freedom of Information Act requests for test essays was unconstitutional on its face in light of the recent transition to computer-based testing in Virginia.  But that judge too went out of his way to deny jurisdiction, this time extending just the opposite reasoning to deny a hearing on its merits.  This time, the court said that I lacked standing.  In other words, it was the court's opinion that I was not applying the case enough to my own injury and so could not bring it on behalf of all the others.  It was a true Catch-22.  Again, this was based on a false premise but nevertheless the only way for the court to ensure the evidence and witnesses were not heard in open court.

     Because of the high importance of the integrity of bar admissions and the novel issues at play with technology, due process, and movement towards more transparent policies nationwide to protect the applicants, I brought a petition before the U.S. Supreme Court to correct the antequated policies of the 7 hold-out states, including Virginia, where applicants have no recourse when a software malfunction affects their test essays.  Certiorari was not granted for that.  Unfortunately, the Court can only hear a mere 2% of its cases, though it is widely known that is only a small fraction of the many cases that have actual merit.

     So we see the Judiciary clearly has its limitations when it comes to software related disputes.  I have no doubt that something went wrong with my essays, and the Board's having fought tooth and nail to keep me from seeing them, for years, says something in and of itself.  To this day, the essays remain saved on an encrypted file held on my laptop, the key to which remains in the hands of the Board.  Had my test been administered in Alaska, for example, the essays would have been sent to me upon request for a charge of $10; in Florida it would have been $50. 

     As Dr. Castell put it well, under these circumstances where a dispute presents itself, the essays themselves represent sine qua non evidence i.e. no chance of bringing a claim that could withstand a motion to dismiss for lack of evidence.  A healthy judicial system must be willing to address technology and people's rights, even if it is not a high dollar case, because it goes to the core of individual liberty.  Short of this, rights will be trampled by those who take advantage of the courts' limited understanding of technology and unwillingness to deal with the obvious implications for citizens' rights.  This will inevitably lead to an unjust result for the individual, which in this case impacts not only the candidates but the public at large.

     There are many benefits to technology.  But as technology increases, transparency must increase in like measure even if it means additional procedural safeguards are put in place.  That is the only way to protect the rights of the individual and our freedom in today's Information Age (*footnote).

*Footnote:  Particularly when evidence derived from Information Technology may arguably be 'ontologically unreliable', as noted earlier in this Journal by Dr. Castell - see for example 'A computer of the simplest kind...', Letter, page 158, [1994] 10 CLSR, May - June 1994, and its appended Footnotes.

VIRGINIA BAR EXAM: LACK OF TRANSPARENCY

Update (31):  I have written an article, along with Stephen Castell PhD, the expert in this case, that was peer reviewed and published in this quarter's edition of the Computer Law & Security Review (Volume 29, Issue 4 (2013), p. 446-449, Computer Based Testing).  I will post it to this site shortly. 

Dr. Castell has become a leading international expert in his field and has testified in American federal courts for both plaintiff and defense, and for many major clients worldwide. 

Update (30):  May 11, 2013.  As a brief update, steps have recently been taken to refer this issue to the legislature of the Commonwealth of Virginia.  At the start, when I was granted an emergency hearing, it quickly became clear that this matter would not be resolved on an emergency basis, which would have saved that career position by simply allowing me to obtain my essays.  The question then became, what position do the courts take on such a matter?  I then began to embark on a course to find out.  Who else could take this course but someone like myself with such a unique position to have standing?  The judicial system, in order to make its rulings on constitutionality, must have before it an injured party to demonstrate why there is a constitutional violation (in this case a due process violation).  The courts are now on record as having dismissed the case without hearing it on its merits.