Insight · In-house interviews
Gimmick or desk skill? What a New York general counsel can actually test.
The interview hour holds two or three finalists. The desk already needs every lawyer who will use a sanctioned tool to catch a bad output. Confusing those two counts is how a fluency line on the spec becomes a parlor trick.
The interview is a count. The desk is a different count.
Of 1,675 structured interviews with New York in-house lawyers, 286 general counsel, deputy general counsel and heads of legal at companies with ten or more attorneys, over a 24-month window, told Sartori they could sit prompt engineering as a live exercise with at most two candidates per open counsel seat.
The room measures performance. The desk will measure whether this hire catches a fabricated citation on a Tuesday. Those are not the same test. Gimmick — drop it as a gate.
A general counsel can test the people who will sit in the open chair. The rest of the desk is a training problem, not an interview. The count is laid out below.
- 32%
- expect attorney headcount to rise135 large corporate legal departments, median revenue $13 billion
- Harbor 2025 Law Department Survey, 8 December 2025
- 85%
- already run an AI resource or committeesame Harbor 2025 sample — governance seats, not new lawyer seats
- Harbor 2025 Law Department Survey, 8 December 2025
- 30%
- of chief legal officers plan to hire more lawyers772 CLOs, 20 industries, 48 countries; nearly 50% at the largest companies
- ACC / FTI 26th Annual CLO Survey, 28 January 2025
- 74%
- of professionals use AI several times a week1,816 professionals across 62 countries, fielded March–April 2026
- Thomson Reuters Future of Professionals Report 2026, 22 June 2026
How many lawyers can you actually test for prompt engineering?
A general counsel can test the open chair. The rest of the desk is a training problem. The public surveys measure the second count. The interview is the first.
Two finalists is an interview. A department already in the tens is a training plan. A New York general counsel who collapses those two numbers onto a single scorecard is not testing a skill. That person is inventing a census the calendar will not hold.
Sartori’s research program maps nearly 1.5 million lawyer profiles globally and has run quarterly market surveys since 2019. The New York in-house findings on this page sit on that instrument. We have worked the New York in-house market for more than ten years, for listed issuers, sponsor-backed platforms, asset-manager legal desks and DFS-regulated financials. Over the trailing three years the New York in-house files ran on a four-to-seven-month clock, with a median of 16 working days from offer to signature. The unit of that work is the open seat. It is not the department.
The Bureau of Labor Statistics counted 90,980 lawyers in the New York-Newark-Jersey City metropolitan statistical area in the May 2023 Occupational Employment and Wage Statistics, at an annual mean of $213,420. That is the wage-and-salary stock across firms, companies and government. It is not a count of people a chief legal officer can put through a live writing exercise on Tuesday. BLS does not publish a Standard Occupational Classification for “prompt engineer”; the Handbook still prices lawyers (national median $151,160 as of May 2024, 864,800 jobs in 2024) and, if the company hires a non-lawyer builder, software developers (national median $133,080 as of May 2024). There is no public pay band for the skill the spec is reaching for. The comparable measured bands are the in-house title the hire will hold, and the developer seat if the company is actually buying a builder.
CLOC’s 2025 State of the Industry report, built on Harbor’s 2024 wave of 186 organizations, put overall median attorney headcount at 44, on a median legal department of 77 FTEs. Attorney headcount climbs with company revenue; the bands sit in the chart below. A general counsel filling one counsel seat against a desk that already numbers in the tens is not short of lawyers to “test.” That person is short of interview hours. The Thomson Reuters / Georgetown 2026 Report on the State of the US Legal Market, released 7 January 2026, recorded that nearly 90 percent of general counsel report resource limitations preventing the strategic impact their organizations expect. The scarce resource is the hour, not a prompt-syntax census.
Of those 286 general counsel, deputy general counsel and heads of legal in the same New York cohort, 204 said every practicing lawyer who would use a sanctioned tool needed to be able to verify output. Only 41 said they would score prompt authorship as a gate on a commercial or contracts counsel hire. That is the split this page is about. The first number is a duty the ethics opinions already impose. The second is a craft that belongs on a handful of hybrid seats. One hundred and twelve of the same 286 said they had never sat a live writing exercise with a counsel candidate at all. The fluency line on the spec is running ahead of the interview the company has actually designed.
Of 24 closed New York in-house searches over the trailing three years, 9 were commercial or contracts counsel, 6 privacy or cybersecurity, 4 funds or product-regulatory, 3 employment or HR legal, and 2 legal-operations or knowledge hybrids. A scored writing exercise appeared in 3 of those 24 files: both hybrids, and one privacy seat where the general counsel wanted to watch the candidate refuse to paste a DPIA into a public model. The other 21 files were practicing-lawyer seats. Syntax was not the work. Judgment was.
A general counsel at a New York-headquartered global asset manager told Sartori they could sit two finalists with a live writing exercise, and that scoring the rest of the desk was a fantasy of the requisition, not of the calendar. That is the testimony that matches the count. The in-house and general counsel search we run is built on that unit: one open chair, a designed exercise, a human score. It is not a department-wide certification.
What the interview can holdWhat the desk already needs
- Open-seat finalists Two or three people the general counsel can actually sit. This is the only population an interview tests.
- The AI governance node Usually legal operations or a dedicated committee. They design the tools. They are not a new counsel headcount line.
- Every lawyer who will use the tools Verification, confidentiality, candor. A training and supervision duty, not an interview census.
A general counsel can test the people who will sit in the open chair. The rest of the desk is a training problem, not an interview.
The skill the desk already needs is verification, not syntax.
New York did not pass a prompt statute. It passed a verification-and-governance stack. A one-shot parlor trick under-tests the duty a chief legal officer is already on the hook for.
The duty is to catch a bad output. A clever prompt is not that duty. ABA Formal Opinion 512, issued 29 July 2024 by the Standing Committee on Ethics and Professional Responsibility, tells lawyers using artificial intelligence tools that they must meet competence, confidentiality, communication, supervision, meritorious claims, candor toward the tribunal, and reasonable fees. Competence includes understanding the benefits and risks of the specific tools. It does not require the lawyer to become an expert in them. Generative tools, in that opinion, are systems that generate new content in response to a user’s prompts and questions. The opinion is Model Rules guidance, persuasive in New York. It is a supervision duty. It is not a syntax exam.
New York City Bar Formal Opinion 2024-5, issued 7 August 2024, applies New York Rules 1.1, 1.6, 3.3 and 5.1 through 5.3 to the same tools. It requires independent verification of AI-generated citations and legal assertions, protection of confidential data, vendor due diligence, and supervision of subordinates and non-lawyer staff. The interview analog is catching a bad output, not producing a pretty one. A head of legal at a DFS-regulated insurer told Sartori they needed someone who could interrogate a model under Circular Letter 7, not someone who could write a clever prompt. That is the testimony that matches the ethics stack.
Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 22 June 2023), is the demonstration case every New York in-house litigator already knows. Counsel filed an opposition citing nonexistent judicial opinions produced by a generative chatbot, then stood by the citations after the court questioned them. Sanctions of $5,000, joint and several, into the court registry, plus letters to the client and to judges falsely named, were grounded in Rule 11 and bad faith. The court did not ban prompting. It punished the failure to verify. The interview that maps to Mata is a supervised review of AI-drafted authority, sat with the mid-level lawyer who will be the first reader, with the general counsel accountable for supervision.
22 NYCRR Part 161, added by Administrative Order AO/75/26 of 25 March 2026 and effective 1 June 2026, applies to all Unified Court System courts, civil and criminal. The policy is that AI tools in preparing court papers should not be prohibited, and that attorneys and parties should not be required to disclose AI use on filing, because existing duties already apply. Courts may adopt a model part rule under which a signature certifies independent review that the paper contains no fabricated or fictitious cases, statutes, or other material. Sanctions remain available under 22 NYCRR 130-1.1 and New York Rule of Professional Conduct 3.3. Pending disclosure bills were not law as of the research window. The skill the Unified Court System is policing, from June 2026, is independent review. It is not disclosure theater, and it is not prompt authorship.
For insurance and bank legal desks the stack is thicker still. NYDFS Insurance Circular Letter No. 7, issued 11 July 2024, tells New York-authorized insurers to develop and manage artificial-intelligence systems and external consumer data in underwriting and pricing so use complies with unfair-discrimination rules. Senior management is responsible for policies, competent staff, model-risk management, independent challenge, internal audit and remediation. The 16 October 2024 industry letter folds AI-related threats into 23 NYCRR Part 500 risk assessments. The 21 May 2026 frontier-model advisory asks CISOs of covered entities for updated assessments, human oversight of AI-generated code before production, and heightened monitoring — and explicitly does not impose new Part 500 requirements. The hire this work wants is a mid-to-senior regulatory counsel who can interrogate a model. It is not a junior technician of prompts.
Of 164 New York in-house hiring managers in the same cohort who answered the 2026 second-quarter screen-design question on Sartori’s quarterly survey — the survey has run since 2019 — 132 ranked catching a fabricated citation above writing a production prompt as a screen for practicing-lawyer seats; 32 ranked prompt authorship first. Four to one. That is the instrument talking, not a slogan. Thomson Reuters’ Future of Professionals Report 2026, fielded March through April 2026 among 1,816 professionals in 62 countries, found 62 percent saying access to professional-grade AI would factor in accepting a new role. That access number is why a chief legal officer who refuses sanctioned tools will lose people for a different reason than a parlor trick. Confidentiality still belongs in the exercise, because unsanctioned tools are already on desks — the share sits in the chart in the next section.
What the room is tempted to scoreWhat the rule actually names
- Competence Opinion 512: understand benefits and risks of the specific tools. Not a requirement to become an expert in them.
- Confidentiality Rule 1.6 and vendor diligence. The exercise should include what must not be pasted into a public model.
- Candor and review Mata, City Bar 2024-5, Part 161. Independent verification of citations and legal assertions. This is the skill.
- Hiring-tool law Local Law 144. An automated scorer of candidates is a different file from a human-scored review.
| Design | What it measures | Maps to | Gimmick or skill |
|---|---|---|---|
| Parlor trick | Fluency in a room | No ethics opinion names this as the duty | Gimmick on a practicing-lawyer seat |
| Automated ranker | A tool’s score of prompt quality | Local Law 144 if it substantially assists the decision in New York City | A compliance file, not a skill test |
| Supervised review | Catching fabricated citations, bad clauses, leaky prompts | Opinion 512; City Bar 2024-5; Mata; Part 161 | Skill — sit it with finalists |
| Production-prompt authorship | Prompts, schemas, evaluation suites, workflows | CLOC Technology and Knowledge Management | Skill — on the ops / knowledge hybrid only |
The parlor trick
One clever prompt, scored for fluency. It measures performance in a room. It does not measure whether the hire will catch a fabricated citation on a Tuesday.
The automated ranker
A tool that scores prompt quality and ranks the slate. In New York City that is an automated employment decision tool before it is a skill test.
Supervised review
A live, human-scored review of AI-drafted authority, a playbook markup, or a model-risk memo. This is the duty Formal Opinion 512 and Part 161 actually name.
Production-prompt authorship
Prompts, schemas, evaluation suites and workflows as the work itself. Belongs on a legal-operations or knowledge hybrid, not on every commercial counsel seat.
The duty is to catch a bad output. A clever prompt is not that duty.
The buyer is a general counsel filling a titled seat, not a prompt occupation.
Prompt authorship does not appear as a job family on the corporate title ladder. The interview is a test inside Attorney through Associate General Counsel, or on a legal-operations hybrid. Those are different scorecards.
The Association of Corporate Counsel’s 2025 Chief Legal Officers Survey — 772 CLOs, 20 industries, 48 countries, released with FTI Consulting on 28 January 2025 — is a corporate-department sample. Seventy percent of chief legal officers manage at least two additional functions (risk, compliance, privacy, ethics). Eighty-three percent of U.S. chief legal officers report to the CEO. Understaffing is the top barrier. Forty-four percent planned to adopt new legal technology in the next year. That is the employer type that would decide whether a writing exercise belongs in an interview for a practicing lawyer. It is not a law-firm hiring committee.
ACC and Empsight’s 2025 Law Department Compensation Survey, data effective 1 March 2025, published 16 September 2025, priced 29 in-house job titles across 1,632 U.S. incumbents. The families a general counsel actually interviews are the ones on that ladder: General Counsel / Chief Legal Officer; deputy; Associate General Counsel; Managing Attorney; Senior Attorney; Attorney; and the legal-operations rung from analyst through vice president. Prompt authorship does not appear as a family. The interview this page is about is therefore a test inside one of those titles, most often Attorney through Associate General Counsel, or a legal-operations manager or director. It is not a new occupation.
Sartori maps roughly 67,000 lawyers in the New York market. That figure is coverage for a search, not a claim about who is available to sit a writing exercise. The seats that actually posted in this cycle, on employer careers pages, split three ways. In-house Legal Counsel, Vice President and Director seats at New York asset-manager and bank legal desks own products that run on models and do not say “prompt” in the posting: a Vice President, private-equity transactions counsel, New York, posted in March 2026, at $190,000 to $235,000 plus discretionary bonus, asked for a J.D. and four-plus years at a major firm or asset-manager legal department; a Director, digital-assets legal counsel, posted in June 2026, at $255,000 to $300,000, asked for eight-plus years and product-regulatory judgment on tokenized cash. Public-body attorneys at the New York City Law Department, the State Attorney General and the Department of Financial Services hire on bar admission and bureau skill. The City’s experienced-attorney bands, accessed September 2026, run $87,737 to $122,300 at 0 to 5 years through $180,791 to $188,235 at 20-plus years; the entry class is typically 40 to 80 recent graduates a year into litigating divisions. Prompt authorship is not a listed criterion on those pages. DFS posted, in the same window, Virtual Currency attorneys in New York City and Assistant Director seats for policy and operations in the Responsible Artificial Intelligence Safety and Education unit inside DIGIT. Those are product and policy files. They are not a prompt occupation.
CLOC’s Core 12 is the competency map corporate legal-operations teams actually use. Technology is defined as automating processes, digitizing tasks and deploying solutions; Knowledge Management as collecting and structuring knowledge so work is easier. A writing exercise on the company side, if it exists, is more likely to be designed by those nodes than by a practice lead. CLOC announced on 19 August 2026 an AI Intensive, inaugural in-person, for about 40 participants on 27 October 2026 in New York City, with real-time prompting exercises, workflow and agent design, and implementation planning, aimed at legal-ops teams moving from pilots to enterprise deployment. That is a measured signal of who the market thinks needs prompt fluency: the operations and innovation bench, in a small cohort, not every Attorney on a tens-of-lawyers desk. The legal operations search is the mandate on which a scored production-prompt exercise is the job.
Anyone who will give legal advice in New York still needs a J.D. and either New York admission or registration under 22 NYCRR Part 522. The 15 April 2020 amendments dropped foreign-bar reciprocity, extended the registration window to 90 days, and kept the registration as permission to serve a single non-law-firm employer, not as admission to the New York bar. IAPP’s CIPP/US and AIGP certificates are optional fluency signals on privacy and AI-governance seats. They are not licenses and they are not a substitute for the bar. AIGP covers AI systems, applicable laws and the AI life cycle. It does not certify prompt authorship. NIST’s AI Risk Management Framework 1.0 (26 January 2023) and Generative AI Profile NIST-AI-600-1 (26 July 2024) are voluntary federal standards a governance counsel may map a program to. They do not create a New York prompt license.
A minority of New York-eligible seats do write prompt authorship into the job, and they sit on the firm side: Technology & Information Services or Knowledge Management counsel, with posted pay in 2026 in bands around $220,000 to $250,000, $250,000 to $290,000, and $193,000 to $301,000, requiring a J.D., years of practice, and work on prompts, schemas, evaluation suites or prompt libraries. Those seats are context. They are not the corporate general counsel’s direct report, and they are not the spine of this page. Copying that interview onto a commercial counsel hire tests a vendor skill on a lawyer who will be asked for legal judgment. A legal-AI company’s own “Legal Engineer” role, described on 29 April 2026, is a technology-company solutions seat, not a New York bar-advice seat. Do not copy it.
Analyst, Legal Operations
Median base $88,000; median total cash $95,000. Not the practicing-lawyer interview.
ACC / Empsight 2025 Law Department Compensation Survey, data effective 1 March 2025 ↗Score the candidate with a person, or inherit Local Law 144.
A human-scored review is an interview. An automated scorer of prompt quality, used to rank New York City candidates, is an automated employment decision tool. Design the exercise before the first round. Adding it later is how files stall.
Local Law 144 of 2021 took effect on 1 January 2023; the Department of Consumer and Worker Protection began enforcement on 5 July 2023. Employers and employment agencies in New York City may not use an automated employment decision tool unless an independent bias audit was completed within the prior year, a summary of results is publicly available, and notice goes to City-resident candidates or employees at least 10 business days before use. An AEDT is a computer-based tool that uses machine learning, statistical modeling, data analytics or artificial intelligence and that substantially assists or replaces discretionary decision-making in hiring or promotion screening. It applies if the job is in a New York City office, is fully remote but tied to a City office, or the employment agency is in the City. Civil penalties run $500 to $1,500 per day. DCWP enforces the audit and notice rules; discrimination claims go to the City Commission on Human Rights.
A general counsel sitting across the table with a red pen is not an AEDT. An automated ranker of prompt quality is. The interview design is therefore a Local Law 144 question before it is a skill question. The State Comptroller’s audit of DCWP’s enforcement, Report 2024-N-6, December 2025, covering July 2023 through June 2025, found DCWP’s 311 complaint routing ineffective; despite only two AEDT complaints in the period, DCWP did not test whether intake worked. DCWP surveyed 32 company websites and found one non-compliance issue; auditors reviewing the same set identified at least 17 potential violations. Companies that automated any slice of legal hiring in New York City should expect counsel time on audit design and website summaries in 2026. They should not expect a new prompt-occupation title to appear because of it.
Among larger companies, 26 percent of chief legal officers in ACC’s 2025 CLO Survey cited concern about AI regulatory enforcement, behind antitrust at 41 percent in that cut. Twenty-three percent of organizations had been subject to a regulatory investigation or enforcement action in the prior year. The RAISE Act, signed 19 December 2025 and effective 1 January 2027, covers frontier-model developers with more than $500 million in revenue, creates an oversight office inside DFS, and gives the Attorney General civil penalties of up to $1 million for a first violation and $3 million for a subsequent one, on IAPP’s 22 December 2025 read of the signed text. That is a product and regulatory file at covered developers, plus the DFS RAISE seats already posted in New York City. It is not a mandate to test every corporate counsel candidate on prompt syntax.
The capital cycle on the tools is moving faster than the job architecture. Harvey raised $200 million at an $11 billion valuation on 25 March 2026, as Reuters reported that day, and $550 million at a $15.5 billion valuation on 9 September 2026. Those rounds buy agents and vendor legal-engineering teams. They do not create a Census code, and they do not open attorney seats. Harbor’s 2025 sample is the other half of that sentence: departments are buying governance faster than they are hiring lawyers.
The finding that does not flatter us sits in the mandate file. In 5 of the 9 commercial and contracts searches, the company added an unsourced “AI fluency” screen after the first round. Three of those five required a second shortlist, because the parlor trick filtered out commercial lawyers who had never authored a production prompt and who were otherwise the hire. Those five files ran to the long end of the four-to-seven-month window. The two legal-operations hybrids that designed the exercise before the first round closed inside four months. We had to rerun slates we should have underwritten against a designed exercise, not against an adjective. New York in-house counter-offer incidence on this line is 28 percent; the candidates who drew them were the ones who had already sat a competing company’s verification exercise, not the ones who had written a clever prompt in ours.
Design the exercise before the first round, or do not add it later.
Month one of the search is when the scorecard is cheap to rewrite. After round one it is a second shortlist. The protocol below is an operating checklist, not a certification scheme.
Design the exercise before the first round, or do not add it later. The cheaper intervention is a one-page scorecard the general counsel, the head of legal operations and the head of talent sign before anyone calls a candidate. For a practicing-lawyer seat the exercise is a supervised review: an AI-drafted opposition with at least one fabricated citation, a playbook markup the tool missed three clauses on, or a model-risk memo that pastes confidential pricing into a public model. The candidate works in the room. The general counsel scores with a red pen. No tool ranks the slate. For a legal-operations or knowledge hybrid the exercise is the work: author, evaluate and iterate a production prompt against the company playbook, with an evaluation suite. Same human score. Different craft.
Thomson Reuters estimated, in the 2026 Future of Professionals release of 22 June 2026, a $232,000 replacement cost per professional among those living inside the value gap. That figure is a survey translation, not a BLS series, and it is useful only as a reminder that a bad screen which loses a hire you already wanted is not a free option. Three composites from the New York in-house file show the same math at mandate scale.
Commercial counsel, listed issuer
A New York head of legal added an unsourced fluency screen after round one. Strong contracts lawyers washed out. Second shortlist. File ran to seven months.
Model-risk counsel, DFS-regulated insurer
The exercise was a Circular Letter 7 interrogation of an underwriting model, scored by the general counsel with a red pen. No syntax test. The seat filled on the existing clock.
Knowledge hybrid, legal operations
Production-prompt authorship was the job. A scored writing exercise belonged in round one, designed with the ops lead, closed inside four months.
The first composite was a commercial counsel seat at a New York listed issuer. The head of legal added an unsourced fluency screen after round one. Contracts lawyers who could mark a playbook washed out because they had never authored a production prompt. We rebuilt the slate. The file ran to seven months. The second was model-risk counsel at a DFS-regulated insurer. The exercise was an interrogation of an underwriting model under Circular Letter 7, scored by the general counsel. No syntax test. The seat filled on the existing clock. The third was a knowledge hybrid reporting to the head of legal operations. Production-prompt authorship was the job. The exercise belonged in round one and the file closed inside four months. None of those companies is named here. The pattern is the product.
| Seat | What to put in the room | Local Law 144 | Ethics hook | Who scores |
|---|---|---|---|---|
| Commercial / contracts counsel | Verify an AI-drafted playbook markup against the company form; name three clauses the tool missed | Human-scored: not an AEDT | Opinion 512 competence; confidentiality of the form | General counsel |
| Privacy / cyber counsel | Review an AI summary of a vendor DPIA; flag what must not be pasted into a public model | Human-scored: not an AEDT | Rule 1.6; DFS Part 500 vendor AI risk | Head of legal + CISO legal support |
| Funds / product-regulatory counsel | Interrogate a tokenized-product memo the tool drafted; state what remains a legal judgment | Human-scored: not an AEDT | Opinion 512; product-regulatory ownership | General counsel |
| Employment / HR legal | Walk a live candidate notice under Local Law 144; say when a tool becomes an AEDT | Do not auto-score this candidate | Local Law 144 itself is the file | Head of legal + head of talent |
| In-house litigator (SDNY / NY state) | Catch fabricated citations in an AI-drafted opposition; certify independent review | Human-scored: not an AEDT | Mata; Part 161; Rule 3.3 | General counsel / deputy |
| Legal-ops / knowledge hybrid | Author, evaluate and iterate a production prompt against a playbook, with an evaluation suite | Human-scored unless a tool ranks the slate | CLOC Technology / Knowledge Management craft | Head of legal operations |
For a general counsel filling Attorney through Associate General Counsel, the exercise is verification, sat with two finalists.
- Write the scorecard first. Independent review of AI-drafted authority, playbook markup or a model-risk memo. Name the three things that fail the candidate: standing by a fabricated citation, pasting confidential matter into a public model, inability to say what remains a legal judgment.
- Score it with a person. The general counsel or deputy, in the room, with a red pen. An automated ranker of prompt quality in New York City is a Local Law 144 file.
- Do not gate on production-prompt authorship. That craft lives on the ops hybrid. Using it here is how commercial lawyers wash out and the file runs to seven months.
- Train the desk separately. Part 161 and Opinion 512 already bind the lawyers on payroll. That is a supervision program, not a second interview.
For a head of legal operations filling a knowledge or workflow hybrid, the exercise is the work itself.
- Sit production-prompt authorship in round one. Prompts, schemas, evaluation suites, playbooks, end-to-end workflows. CLOC’s Technology and Knowledge Management nodes are the design owners.
- Still score it with a person. A tool that ranks the slate in New York City trips Local Law 144 even on an ops hire.
- Keep the bar gate honest. If the seat will give legal advice, it still needs a J.D. and New York admission or Part 522 registration. AIGP is a fluency signal, not a license.
- Do not copy a vendor Legal Engineer interview. That is a technology-company solutions seat. This hire will sit inside the company’s legal function.
Questions a general counsel asks before adding a writing exercise
Should a general counsel put prompt engineering in a legal interview?
Yes — with the two or three finalists for the open seat, not as a census of the desk. Of 1,675 structured interviews with New York in-house lawyers, 286 general counsel, deputy general counsel and heads of legal over a 24-month window told Sartori they could administer a live writing exercise to at most two candidates per open counsel seat. Score whether the hire can catch a bad output under ABA Formal Opinion 512 (29 July 2024) and 22 NYCRR Part 161 (effective 1 June 2026). A parlor-trick prompt that does not test candor or confidentiality is a gimmick.
How many lawyers on an in-house desk can a company actually put through a live prompt exercise?
The people competing for the open chair — typically two finalists — not the rest of a tens-of-lawyers department. CLOC’s 2025 State of the Industry report, from Harbor’s 2024 wave of 186 organizations, put overall median attorney headcount at 44; Harbor’s 2025 wave of 135 departments found only 32 percent expecting to add attorneys. The interview hour does not scale to 44 people. Verification of sanctioned tools is a training and supervision problem for the lawyers already on payroll.
Does New York require in-house lawyers to pass a prompt test?
No New York statute requires a prompt-writing test as a condition of practicing on a corporate desk. ABA Formal Opinion 512 (29 July 2024) says lawyers using generative tools must understand benefits and risks; they need not become experts in the tools. 22 NYCRR Part 161, effective 1 June 2026, allows AI in court papers, refuses a disclosure obligation, and keeps fabricated citations sanctionable. The skill the Unified Court System is policing is independent review, not syntax.
When does a scored writing exercise become a Local Law 144 problem?
When a computer, not a person, substantially assists or replaces the hiring decision in New York City. Local Law 144 of 2021, with Department of Consumer and Worker Protection enforcement from 5 July 2023, requires a bias audit within the prior year, a public summary, and at least 10 business days’ notice to City-resident candidates, with civil penalties of $500 to $1,500 per day. A general counsel sitting across the table with a red pen is not an automated employment decision tool. An automated ranker of prompt quality is.
What should a chief legal officer test instead of a parlor-trick prompt?
Whether the candidate can independently verify an AI-drafted authority, a playbook markup, or a model-risk memo, and say what must not be pasted into a vendor tool. New York City Bar Formal Opinion 2024-5 (7 August 2024) requires independent verification of AI-generated citations and legal assertions. Mata v. Avianca, 678 F. Supp. 3d 443 (S.D.N.Y. 22 June 2023), sanctioned counsel $5,000 for standing by chatbot citations. That is the desk skill. Prompt syntax is a legal-operations craft on a handful of hybrid seats.
Does a legal-operations or knowledge hire need a different interview from counsel?
Yes: sit production-prompt authorship on the ops hybrid, not on every counsel seat. CLOC announced on 19 August 2026 an AI Intensive in New York City on 27 October 2026 for about 40 legal-operations practitioners, with real-time prompting and agent design. That is the scale of the ops skill-build. A practicing-lawyer seat is still Attorney through Associate General Counsel on the ACC / Empsight 2025 title ladder, data effective 1 March 2025. Put production-prompt authorship on the ops scorecard; put verification and confidentiality on the counsel scorecard.
BLS, Harbor, ACC, the Unified Court System, and the New York interview file.
Lawyer stock and the missing occupation code come from BLS. Desk size and AI-committee shares come from Harbor and CLOC. In-house titles and pay come from ACC and Empsight. The ethics stack is Formal Opinion 512, New York City Bar Formal Opinion 2024-5, Mata v. Avianca, and 22 NYCRR Part 161. Local Law 144 and the Comptroller audit govern automated scoring in New York City.
Sources and further reading
40 references- Sartori & Partners — New York Legal Talent Research Programme (1,675 structured interviews; ~67,000 lawyers mapped; quarterly surveys since 2019; mandate telemetry) sartoriglobal.com ↗
- New York-Newark-Jersey City, NY-NJ-PA - May 2023 OEWS Metropolitan Occupational Employment and Wage Estimates bls.gov ↗
- Lawyers : Occupational Outlook Handbook : U.S. Bureau of Labor Statistics bls.gov ↗
- Software Developers, Quality Assurance Analysts, and Testers : Occupational Outlook Handbook bls.gov ↗
- 2025 Law Department Compensation Survey Executive Summary (ACC / Empsight) acc.com ↗
- Association of Corporate Counsel and Empsight Release Industry-Leading, Comprehensive Data Report for In-house Professionals acc.com ↗
- 2025 ACC Chief Legal Officers Survey Key Findings acc.com ↗
- The Association of Corporate Counsel and FTI Consulting Release Findings From Annual Chief Legal Officers Survey fticonsulting.com ↗
- CLOC 2025 State of the Industry Report (2024 Harbor Law Department Survey) cloc.org ↗
- Harbor 2025 Law Department Survey Reveals Surge in AI Integration, Falling Outside Counsel Spend harborglobal.com ↗
- CLOC Releases 2026 State of the Industry Report cloc.org ↗
- CLOC's Core 12 framework cloc.org ↗
- CLOC Launches 'AI Intensive' cloc.org ↗
- Future of Professionals Report 2026 (Thomson Reuters) datocms-assets.com ↗
- AI is Ready but Firms are Not (Thomson Reuters / PR Newswire, 22 June 2026) prnewswire.com ↗
- Legal Industry Experiencing Tectonic Shift (Thomson Reuters, 7 January 2026) thomsonreuters.com ↗
- 2026 Report on the State of the US Legal Market (Thomson Reuters / Georgetown) blogs.thomsonreuters.com ↗
- Formal Opinion 512 (29 July 2024) — Generative Artificial Intelligence Tools americanbar.org ↗
- Formal Opinion 2024-5: Generative AI in the Practice of Law (New York City Bar) nycbar.org ↗
- Effective June 1, 2026, The New York State Unified Court System Has Adopted a New Rule Regarding the Use of Artificial Intelligence nysba.org ↗
- Part 161. Use of Artificial Intelligence Technology (New York State Unified Court System) nycourts.gov ↗
- Opinion and Order on Sanctions (Mata v. Avianca, S.D.N.Y. 22 June 2023) nhd.uscourts.gov ↗
- Automated Employment Decision Tools (AEDT) - New York City DCWP nyc.gov ↗
- Automated Employment Decision Tools: Frequently Asked Questions (DCWP) nyc.gov ↗
- Department of Consumer and Worker Protection: Enforcement of Local Law 144 (Report 2024-N-6, December 2025) osc.ny.gov ↗
- Insurance Circular Letter No. 7 (2024) — NYDFS dfs.ny.gov ↗
- Cybersecurity Risks Arising from Artificial Intelligence (NYDFS, 16 October 2024) dfs.ny.gov ↗
- Heightened Cybersecurity Risks Associated with Frontier AI Models (NYDFS, 21 May 2026) dfs.ny.gov ↗
- Governor Hochul Signs Nation-Leading Legislation to Require AI Frameworks for AI Frontier Models governor.ny.gov ↗
- Hochul enacts New York's AI safety and transparency bill (IAPP, 22 December 2025) iapp.org ↗
- AI Risk Management Framework | NIST nist.gov ↗
- Admitted Attorneys - New York City Law Department nyc.gov ↗
- Entry-Level Attorney Positions - New York City Law Department nyc.gov ↗
- Attorney job postings | New York State Attorney General ag.ny.gov ↗
- Careers with DFS dfs.ny.gov ↗
- New York Adopts New More Workable Registration Requirements for Foreign In-House Counsel pillsburylaw.com ↗
- AIGP: Artificial Intelligence Governance Professional (IAPP) iapp.org ↗
- CIPP/US Certification (IAPP) prod.iapp.org ↗
- Legal software firm Harvey valued at $11 billion in latest funding round (Reuters, 25 March 2026) reuters.com ↗
- Harvey Raises $550M at a $15.5B Valuation to Help Legal Teams Own Their Intelligence harvey.ai ↗
Harbor and CLOC percentages describe participating large departments, not a census of New York City interviews. ACC compensation is incumbent-reported as of 1 March 2025. BLS metro lawyer counts are the May 2023 OEWS establishment survey. Thomson Reuters replacement-cost and value-gap figures are survey translations, not a payroll series.
Write the scorecard. Then fill the chair.
The writing exercise sits on top of a general counsel mandate, a legal-operations function, and a clear read of which skills actually compound. These are the adjacent pieces.
What companies look for in a new general counsel
Succession criteria for a second general counsel search — board access, the compliance book, and the regulator that actually reaches this company — before anyone writes a fluency line on the spec.
Read the succession guideAI and legal careers: skills that increase market value
Verification, governance and judgment as the skills that compound. This page is the interview test; that page is the career texture around it.
Read the skills mapLegal operations career path
The corporate function that actually owns production prompts, schemas and evaluation suites — the seat where a scored writing exercise is the job, not a gimmick.
Read the ops pathFor general counsel and heads of legal
Need a candid read on whether the exercise belongs in the interview — or on the ops scorecard?
We help companies underwrite in-house counsel and legal-operations seats in New York before a fluency line on the spec becomes a second shortlist. Quiet, evidence-led, and just as willing to strike the line as to design the exercise.