What happened?

Connecticut judge Walter Spader Jr. identified the first US case in which a plaintiff embedded hidden commands in court filings meant only for AI systems to read. Plaintiff Matthew Elliott, who claimed a healthcare provider had unfairly denied him access to records, added text in tiny font and white-on-white coloring that was invisible to the human eye.

This hidden text instructed any AI system that might read the filing to align its output with the plaintiff's arguments, ignore the court's previous denials, and grant the requested remedy. Judge Spader said this attempt amounted to a type of prompt injection attack.

Why does it matter?

Judge Spader emphasized that the Connecticut Judicial Branch does not use AI to review filings or make decisions, so the hidden commands posed no real risk in this instance. Still, he described the attempt as 'serious litigation misconduct' and noted that some court systems do use AI to review filings.

Despite a court warning, Elliott continued adding hidden text, some of which included a link to a Nosferatu YouTube video and nonsensical jokes. Spader said this was 'startling' and that it made no sense to add joke-like hidden messages to filings meant to be taken seriously.

What we know

  • The hidden text used small font size and white-on-white coloring to stay invisible to human readers.
  • Elliott argued his intent was to 'audit' the court's AI use, but the judge found this explanation unconvincing.
  • The court did not fine Elliott but barred him from electronic filing in the future.
  • In Brazil, two lawyers who attempted a similar attack on an AI-using court were fined roughly $16,000.
  • The Brazilian system detected the hidden text before processing it, while in Elliott's case the text was discovered when a human reviewed the filing.

What's next?

Judge Spader noted that courts have so far focused on AI output errors, such as hallucinated citations, but that input attacks like prompt injection were 'unanticipated.' Spader argued that courts need to develop new rules to guard against such attacks.

The judge also warned that pro se litigants are misusing chatbots by asking them only for responses that support their own arguments, giving them an unrealistic sense of confidence. Spader said, 'an argument that is prompted only to validate its author is ultimately not even honest with its author.'