California Bar discloses AI was used to develop questions in Exam
By AP Staff
A photo taken on November 27, 2024 shows the logo of the ChatGPT application developed by US artificial intelligence research organization OpenAI on a smartphone screen (L) and the letters AI on a laptop screen in Frankfurt am Main, western Germany. (Photo by Kirill KUDRYAVTSEV / AFP) (Photo by KIRILL KUDRYAVTSEV/AFP via Getty Images)
LOS ANGELES (AP) — The State Bar of California has disclosed that some multiple-choice questions in a problem-plagued bar exam were developed with the aid of artificial intelligence.
The legal licensing body said in a news release Monday that it will ask the California Supreme Court to adjust test scores for those who took its February bar exam.
“The debacle that was the February 2025 bar exam is worse than we imagined,” Mary Basick, assistant dean of academic skills at the University of California, Irvine, Law School, told the Los Angeles Times. “I’m almost speechless. Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable.”
In February, the new exam led to complaints after many test-takers were unable to complete their bar exams. The online testing platforms repeatedly crashed before some applicants even started. Others struggled to finish and save essays, experienced screen lags and error messages and could not copy and paste text, the Times reported earlier.
According to a recent presentation by the State Bar, 100 of the 171 scored multiple-choice questions were made by Kaplan and 48 were drawn from a first-year law students exam. A smaller subset of 23 scored questions were made by ACS Ventures, the State Bar’s psychometrician, and developed with artificial intelligence.
“We have confidence in the validity of the (multiple-choice questions) to accurately and fairly assess the legal competence of test-takers,” Leah Wilson, the State Bar’s executive director, told the newspaper in a statement.
Katie Moran, an associate professor at the University of San Francisco School of Law who specializes in bar exam preparation, told the newspaper, “It’s a staggering admission.”
“The State Bar has admitted they employed a company to have a non-lawyer use AI to draft questions that were given on the actual bar exam,” she said. “They then paid that same company to assess and ultimately approve of the questions on the exam, including the questions the company authored.”
Andrew Perlman, dean of Suffolk University Law School and an advisory council member of the American Bar Association Task Force on the Law and Artificial Intelligence, said he had not heard of AI being used to develop bar exam questions or standards being put in place governing such uses.
But he said he was not surprised, given the rapid growth of AI technology. Perlman said AI can be useful for developing questions for assessment, but a critical guard rail is making sure that everything that comes from an AI tool is vetted carefully by experts in the subject matter.
He expects its use to continue to grow.
Although there might be public skepticism of the emerging technology in the legal profession at this time, “we will be worried in the future about the competence of lawyers who don’t use these tools,” Perlman predicted.
Ten philanthropic foundations are committing $500 million across the next five years to place human interests at the forefront of artificial intelligence's rapid integration into daily life.
Jesse Pickard, CEO of The Mind Company, shares how Elevate and Balance are redefining mental fitness with science-backed tools for brainpower and wellness.
Apple has taken down an app that uses crowdsourcing to flag sightings of U.S. immigration agents after coming under pressure from the Trump administration.
Former Cisco Systems CEO John Chambers learned all about technology’s volatile highs and lows as a veteran of the internet’s early boom days during the late 1990s and the ensuing meltdown that followed the mania. And now he is seeing potential signs of the cycle repeating with another transformative technology in artificial intelligence. Chambers is trying take some of the lessons he learned while riding a wave that turned Cisco into the world's most valuable company in 2000 before a crash hammered its stock price and apply them as an investor in AI startups. He recently discussed AI's promise and perils during an interview with The Associated Press.
Tesla reported a surprise increase in sales in the third quarter as the electric car maker likely benefited from a rush by consumers to take advantage of a $7,500 credit before it expired on Sept. 30. The company reported Thursday that sales in the three months through September rose 7% compared to the same period a year ago. The gain follows two quarters of steep declines as people turned off by CEO Elon Musk’s foray into right-wing politics avoided buying his company’s cars and even protested at some dealerships. Sales rose to 497,099 vehicles, compared with 462,890 in the same period last year.
OpenAI could now be the world’s most valuable startup, ahead of Elon Musk’s SpaceX and TikTok parent company ByteDance, after a secondary stock sale designed to retain employees at the ChatGPT maker. Current and former OpenAI employees sold $6.6 billion in shares to a group of investors, pushing the privately held artificial intelligence company’s valuation to $500 billion, according to a source with knowledge of the deal who was not authorized to discuss it publicly. The valuation reflects high expectations for the future of AI technology and continues OpenAI’s remarkable trajectory from its start as a nonprofit research lab in 2015.
Tom’s Guide Editor-in-Chief Mark Spoonauer breaks down Apple & Amazon's latest product drops—what's hot, what's hype, and what really matters for users.
Police in Northern California pulled over a self-driving Waymo taxi after it made an illegal U-turn. But without a driver behind the wheel, they could not issue a moving violation ticket.
