~ JSQR ~

Why 'Trails'?

Sun, 12 Jul 2026 00:00:00 +0000

The owner of the memex, let us say, is interested in the origin and properties of the bow and arrow. Specifically he is studying why the short Turkish bow was apparently superior to the English long bow in the skirmishes of the Crusades. He has dozens of possibly pertinent books and articles in his memex. First he runs through an encyclopedia, finds and interesting but sketchy article, leaves it projected, Next, in a history, he finds another pertinent item, and ties the two together. Thus he goes, building a trail of many items. Occasionally he inserts a comment of his own, either linking it into the main trail or joining it by a side trail to a particular item. When it becomes evident that the elastic properties of available materials had a great deal to do with the bow, he branches off on a side trail which takes him through textbooks on elasticity and tables of physical constants. He inserts a page of longhand analysis of his own. Thus he builds a trail of his interest through the maze of materials available to him.
And his trails do not fade. Several years later, his talk with a friend turns to the queer ways in which a people resist innovations, even of vital interest. He has an example, in the fact that the outranged Europeans still failed to adopt the Turkish bow. In fact he has a trail on it. A touch brings up the code book. Tapping a few keys projects the head of the trail. A lever runs through it at will, stopping at interesting items, going off on side excursions. It is an interesting trail, pertinent to the discussion. So he sets a reproducer in action, photographs the whole trail out, and passes it to his friend for insertion in his own memex, there to be linked into the more general trail.
—Vannevar Bush (1945). “As We May Think.” The Atlantic Monthly, 176(1), 101-108.

Sources of New York Law

Thu, 05 Feb 2026 00:00:00 +0000

Last updated: February 18, 2026

Pointers and Lists

Cornell LII - New York - List of New York resources from the Legal Information Institute.
New York State Legislative Library - Legislative resources including session laws and local laws.

Statutes

Free Official/Semi-Official Sources

New York State Legislature - Laws of New York - Official unannotated text of the Consolidated Laws and unconsolidated laws. Access via web interface; no API. Updated regularly.
NY Senate Open Legislation - Provides current statutory text through the same platform as bills. Free REST API available (requires registration for API key).

Paid Annotated Codes

McKinney’s Consolidated Laws of New York Annotated (Westlaw/Thomson Reuters) - The primary annotated code used by practitioners. Includes case annotations, practice commentaries, and session law citations. Subscription required.
New York Consolidated Laws Service (CLS) (Lexis) - Alternative annotated code with similar features. Subscription required.

Session Laws

NYS Legislative Library - Laws of the State passed at the sessions of the Legislature.

Bills in Flight

Official Sources

NY Assembly Bill Search - Official Assembly bill tracking. Web interface for searching current and prior session bills.
NY Senate Open Legislation - Primary resource for real-time bill tracking. Offers a free REST API (registration required). Includes bill text, sponsors, committee assignments, votes, and status. Code on GitHub.
Leginfo - Older official interface for bill search, still functional.

Third-Party Tracking

LegiScan - New York - Aggregates NY legislative data with API access. Free tier available; enhanced features require subscription.

Legislative History

Legislative history materials for New York, including: bills, sponsor’s memoranda, governor’s messages, bill jackets, and (sometimes) hearing transcripts.

Research Guides

NYS Library Legislative History Tutorial - Research tutorial from the State Library.
NYU Law - NY Legislative History - Academic research guide.
Brooklyn Law - NYS Legislative History - Detailed guide with finding aids.
Georgetown Law - NY Legislative History - Overview of available documents and locations.

Bill Jackets

Bill jackets contain memoranda, correspondence, and other materials considered by the Governor when signing or vetoing bills. Unique to New York legislative history research.

NYS Archives Digital Collections - Bill Jackets - Free PDF downloads of Governor’s bill jackets from 1995 to present. Searchable by year, chapter, or keyword.
NYS Library Bill Jackets - Holdings for 1905 and 1921-2009 (microform and CD-ROM). Some digitized in library’s digital collections.

Session Laws and Governor’s Messages

Governor’s approval and veto messages are published with the Session Laws and are often the most accessible legislative history documents.

Regulations

Current Regulations

NYCRR on Westlaw and Lexis - Official and annotated NYCRR available on commercial platforms. Subscription required.
Unofficial NYCRR on Westlaw - Free browsable collection.
Cornell LII - NYCRR - Free browsable collection of NY regulations by title. Clean interface.

Proposed Rules and Rulemaking

New York State Register - Official journal of state government published weekly. Contains Notices of Proposed Rule Making and Notices of Adoption. Free online from 2015-present.

Historical Regulations

NYCRR Digital Archive (UB Law Library) - Free access to superseded NYCRR pages (“takeouts”) for 1945-2001 in PDF format.

Agency-Specific

Individual agencies often publish their regulations on their own websites (e.g., DEC Regulations, Education Department).

Judicial Opinions

Official Free Sources

NY Courts - Official Reports - Free access to Court of Appeals and Appellate Division decisions from 1956-present. Slip opinions available same-day for Court of Appeals. Web interface; no public API.
Court-PASS - Court of Appeals Public Access and Search System. Free access to Court of Appeals materials for cases filed after January 1, 2013.
NY eCourts - Case information and filings for state courts. Primarily docket/status information rather than full opinions.
NYSCEF Case Search - New York State Courts Electronic Filing system. Access to filed documents in participating courts.

Free Third-Party Sources

Justia - Free NY case law database.
Findlaw - NY Supreme Court opinions from 1959-present.
Google Scholar - Searchable NY case law. Free but unofficial.

Paid Commercial Databases

Westlaw (Thomson Reuters) - Comprehensive NY case law with headnotes, KeyCite citator, and secondary sources. Subscription required.
Lexis+ (LexisNexis) - Comparable coverage with Shepard’s citator. Subscription required.

Computational Law

Sun, 04 Jan 2026 00:00:00 +0000

Last updated: January 6, 2026

Background: Introduction to Computational Thinking

MIT offers a course called Introduction to Computational Thinking (6.C25 / 18.C25), which is available online. Okay—nothing special so far. There are lots of course videos and lecture notes on the web now. What’s more remarkable is that the entire course—lectures and homeworks—is structured around interactive notebooks mixing explanatory text, worked examples, and exercises.

This is a brilliant way to organize a course. And, of course, it’s not entirely new. Wolfram has been sponsoring books that are essentially Mathematica notebook printouts since at least the early 90s. And the idea of a notebook interface borrows from Knuth’s literate programming. But somehow bundling everything together in a consistent package on the web makes a difference, inviting engagement and experimentation. It takes on the spirit of an upper-level physics lab, where you are given background reading, equipment, aims, and some high-level notes on procedure, but have enough options and flexibility to make things interesting.

I worked through the Fall 2024 version of the course because I was interested in the core topics (epidemic models, climate science, and the Julia programming language). I wonder, though, if the main thing I took away was enthusiasm for this style of teaching technical information.

Law ⊂ Code

Running with the thesis that law is like computer code, or that law is a form of ‘code’ in the broader sense, we can imagine what an analogue of the MIT course might look like. Begin with adapting existing coding agents, just to get a sense of the possibilities of working with text in the same way programmers work with computer languages. Then, progress through a sequence of notebooks, demonstrating approaches to increasingly complex legal tasks.

Along the way, we’d need to explore answers to several questions. Programming languages have compilers (or interpreters), linters, type-checkers, assertions, and test suites; how can we adapt those ideas to legal language? What is the equivalent of formal verification for natural language, and is it worth the trouble? How do we handle editing and versioning? Coding agents are currently a bit like REPLs; what’s the right user interface for other modes of development? Is there something like ‘literate programming’ when the object code is itself a contorted form of natural language?

I wouldn’t expect to come up with definitive answers to those questions, which are both difficult and vague, but it is important to adopt a point of view on at least some of them.

An outline for an introduction to computational law

Very much a work in progress. Exercises would be in the form of interactive notebooks with explanatory text, background references, and some scaffolding and examples already in place to make things more accessible.

Prelude. Reading on the recent history of neural networks, transformers, and language models. Augmentation versus automation: what factors make technology positive or negative, and for which groups? Moving beyond the chatbot: data curation, coding agents & tools. Exercise: develop several agent skills to aid legal problems. Example: extracting citations from a brief along with a summary of the relevant rule of law.
Using Language Models through an API. Legal topic: regulation of the legal profession; unauthorized practice of law. Python and technical infrastructure review. Exercise: set up infrastructure (notebooks, dependency management, API keys) for the rest of the sequence. Practice extracting structured data from text and generating summaries.
Embeddings, classification, and retrieval. Legal topic: statutes, court opinions, and applying legal rules to facts. Exercise: implement a classifier for legal clauses in two ways (few-shot prompting and using kNN and embeddings). Implement a retrieval tool for legal texts.
Working with legal texts. Structural properties of legal texts: outlines, headings, citations, rules, and entities. Exercise: redo the example from first-week exercise (extracting citations and summarizing rules) in computer code.
Evaluation. Legal topic: professional ethics and use of technology (competence; confidentiality, privacy and security). Simple metrics and strategies for evaluating classification, retrieval, and quality of generated output. Exercise: characterize (a) classification performance on a simple labeled dataset, (b) retrieval performance, and (c) performance on generating summaries of legal rules.
Logic, law, and automated reasoning. Classical propositional and predicate logic. Modal logic, defeasible logic, and temporal logic or situation/event calculus. Approaches to automated reasoning. Legal topic: civil law jurisdictions. Exercise: try evaluating legal propositions by ‘faking’ formal logic. Where do you draw the line between capturing relevant domain logic (the logic of a legal rule) and falling back on the language model’s sea of background knowledge? Compare results in toy examples to automated reasoning systems like the Z3 SMT solver or Prolog.
Tabular data. Legal topic: contracts and litigation discovery. Overview of commercial applications. Exercise: build a tabular analysis tool for collections of legal documents, building on the previous exercises.
Ethics & citation-checking. AI ethics: transparency, explainability and interpretability, biases, errors. Ethics of assistive tools for citations. Exercise: build a system that operates over a corpus of interlinked documents, checking citations to other documents in the corpus and checking that the references actually support the claims, and presenting results for human verification. Build a knowledge graph from the citations in a set of cases and a simple ‘Shepardizing’ tool.
Specifications & drafting aids. Legal topic: drafting a legal memo. Exercise: build a tool to aid drafting legal texts. Generated text should be a fragment of a legal document, based on a precedent, satisfying requirements (i.e., checked using a suite of specs or tests), and making claims grounded in cited authorities.
Fine-tuning. Legal topic: Regulation of AI. Basics of fine-tuning models using Axolotl or similar tools. Exercise: reimplement one of the functional elements you’ve built in previous modules (e.g., extracting citations) by fine-tuning a small model. Evaluate task performance as well as resource requirements; can you maintain quality while increasing speed and efficiency on the task?
Orchestration. Legal topic: Government and AI. Compare approaches to orchestrating AI functions: coding agents and skills, programming frameworks, tools & MCP. Exercise: create a legal agent for working with precedents, specifications, and authorities to produce fragments of legal text to serve as first drafts of a memo.

Self-supervised retrieval evaluation

Tue, 09 Dec 2025 00:00:00 +0000

Last updated: December 26, 2025

Evaluation with synthetic queries

Even if you lack real labeled data, it’s still often possible to do meaningful evaluations of retrieval performance using synthetic data.

For example, if you have a piece of text divided into segments and some way to generate a plausible query for which that segment would be a good response, you can create an evaluation dataset by doing this:

loop N times:
    choose arbitrary segment
    query = create_query(segment)
    save (query, segment) to dataset

Synthetic data and property-based testing

When synthetic data is used for training or fine-tuning, it makes sense to think of it as data. But in the context of evaluation, you can also think of the whole process as a kind of stochastic property-based testing, where we verify that some circuit we think ought to exist (based on our understanding of the problem) is in fact closed as expected.

a closed loop

It turns out there’s a whole lore around property-based testing, often exploiting formal properties of domain objects and operations on them. E.g. (following Wlaschin 2014):

commutative relationships (and model-based approaches),
invertible operations,
invariance under transformation,
idempotence, and
structural induction.

The example given above fits the ‘invertible operations’ paradigm, where the operations are ‘given a query, retrieve a responsive segment’ and ‘given a segment, generate a plausible query’.

But really, for lots of properties that involve a generation step you could collect a dataset of generated examples and call it synthetic evaluation data.

References:

Claessen, K. and Hughes, J. (2000). QuickCheck: a lightweight tool for random testing of Haskell programs. ACM SIGPLAN Notices, Vol. 35, Issue 9, pp. 268-279.
Es, S. (2024). All about synthetic data generation. Ragas blog.
Esfandiarpoor, R. et at. (2025). Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevance. arXiv:2503.23239 [cs.IR].
Rahmani, H. (2024). Synthetic Test Collections for Retrieval Evaluation. arXiv:2405.07767 [cs.IR].
Wlaschin, S. (2014). Choosing properties for property-based testing. F# blog.

AI for legal epidemiology

Wed, 12 Nov 2025 00:00:00 +0000

Last updated: December 31, 2025

Epidemiology starts with data

Overview

Legal epidemiology—understanding the effects of laws on health—requires an understanding of what the law is. But laws are spread across jurisdictions and different levels of government, and are framed in technical language that can be difficult to encode for analysis. Since 2024, I have been working with collaborators on a project to apply large language models and modern retrieval methods to make encoding of laws easier, focusing on the municipal codes and ordinances of hundreds of citites in the United States. This is part of a broader interest in making law more accessible.

Project milestones are logged as short updates in the Projects log.

Current version of code

Legiscope.

Materials

Prototype slides (Oct. 7, 2024)

People

Samrachana Adhikari (NYU Langone)
Kimberly Villalobos Carballo (NYU)
Magdalena Cerda (NYU Langone)
Corey Davis (Network for Public Health Law)
Jaskiran Dhinsa (NYU Langone)
Charles DiMaggio (NYU Langone)
Theodore Hill (NYU Langone)
Spruha Joshi (University of Michigan)
Daniel Neill (NYU)
Pooja Shah (NYU Langone)
Utkarsh Srivastava

AI and Legal Informatics

Wed, 12 Nov 2025 00:00:00 +0000

Last updated: March 26, 2026

Stepped Reckoner (Leibniz, 1673)

Background

I came to law school after spending several years doing data analysis and modeling in the physical sciences and finance. In contrast to those fields, legal practice seemed old-fashioned. How could I automate a process? How could I make sense of everything I was reading? Are we still in the nineteenth century or what? For all of its zeal to tackle new problems, law remained in many ways stuck in the past when it came to actually doing the work. It was frustrating, and I even wrote a Note about it as a kind of protest.

We’re now in a different era, and the current crop of transformer-based language models promises to make lots of ideas that were enticingly out of reach a decade or two ago much closer. How can we take advantage of the new AI tools?

Caveat lector: this list is clearly a bit scattered and idiosyncratic. Among other things, it doesn’t include any individual papers. May it improve over time!

AI, general background

Bishop, C. and Bishop, H. (2024). Deep Learning: Foundations and Concepts. Springer. Authors’ site.
Jurafsky, D. and Martin, C. (2025). Speech and Language Processing (3rd ed. draft Aug. 24, 2025). Full draft here.
Russell, S. and Norvig, P. (2021). Artifical Intelligence: A Modern Approach (4th ed.). Pearson. Authors’ site.

Periodicals, conferences, and collections

Organizations and projects

Books on reasoning

Besnard, P. and Hunter, A. (2008). Elements of Argumentation. MIT Press. Full text at ResearchGate.
Bradley, A. and Manna, Z. (2007). The Calculus of Computation. Springer. Manna lecture notes.
Halpern, J. (2017). Reasoning about Uncertainty (2nd. ed.). MIT Press. Full text at MIT Press Direct.
Kroening, D. and Strichman, O. (2017). Decision Procedures (2nd ed.). Springer. Authors’ site.
Kowalski, R. (2011). Computational Logic and Human Thinking. Cambridge Univ. Press. Publisher site.
Mueller, E. (2014). Commonsense Reasoning: An Event Calculus Based Approach (2nd ed.). Elsevier. Publisher site

Books on computational law, generally

Ashley, K. (2017). Artificial Intelligence and Legal Analytics. Cambridge Univ. Press. Publisher site.
Katz, D., Dolin, R. and Bommarito, M., eds. (2021). Legal Informatics. Cambridge Univ. Press. Publisher site.
Lodder, A. and Oskamp, A. (2006). Information Technology & Lawyers. Springer.
Stranieri, A. and Zeleznikow, J. (2005). Knowledge Discovery from Legal Databases. Springer.

Books on ethics & society

Bullock, J. et al., eds. (2022). The Oxford Handbook of AI Governance. Oxford Univ. Press. Publisher site. The print version is expensive, but it should be available in electronic form through institutional libraries.
Crawford, K. (2021). Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale Univ. Press. Author’s site.
Dubber, M., Pasquale, F. and Das, S., eds. (2020). Oxford Handbook of Ethics of AI. Oxford Univ. Press. Publisher site.
Hernández-Cano, A. et al. (2025). Apertus v1 Technical Report. Project site. Report on a family of language models trained by a team at EPFL and ETH Zurich with transparency and other ethical considerations top of mind.
Lessig, L. (2006), Code v2.0. Basic Books. Author’s site with full text. Code may be law, but isn’t the converse also true?
Schneier, B. (2023). A Hacker’s Mind. Norton. Author’s site. Especially the chapters on law and government.
Schneier, B. and Sanders, N. (2025). Rewiring Democracy: How AI Will Transform Our Politics, Government, and Citizenship. MIT Press. Authors’ site.
Vallor, S. (2024), The AI Mirror. Oxford Univ. Press. Author’s site.

Useful tools for hacking LLMs

Axolotl: fine-tune open-weight models.
DeepEval: library for LLM evaluation.
dstack: orchestration for GPU workloads.
Hugging Face Transformers: swiss army knife for working with transformer models.
Instructor: provider-agnostic library for structured LLM outputs.
llama.cpp: local LLM inference (e.g., on a laptop), GGUF format models
Marimo: reactive Python notebooks with bells and whistles.
nupunkt: sentence boundary detection library optimized for legal texts.
NVIDIA DGX Spark: examples and docs for GB10 developer workstations.
Pydantic AI: Python ‘agent’ framework (validation, model abstraction, evals, workflows).
Pytorch: ubiquitous tensor library for deep learning.
Textual: Python TUI application framework (‘move at terminal velocity’).
Typer: Python CLI application library.
uv: a modern Python package manager.
vLLM: LLM inference (e.g., on NVIDIA data center GPUs).
VRAM & Performance Calculator: plan for memory and computation needs.

Data

Free Law Project: case law database (RECAP, CourtListener, etc.).
HLS Caselaw Access Project: scanned law books.
Hugging Face: open models and data sets.
KL3M: free-range LLMs, with training data (lots of legal sources) on Hugging Face.
Pile of Law: dataset on Hugging Face.

Online courses

Guides

The Synthetic Data Playbook: Generating Trillions of the Finest Tokens.