Silviu Pitis

I am a PhD student with the University of Toronto Machine Learning Group and Vector Institute of Artificial Intelligence, under the supervision of Jimmy Ba. I’m primarily interested in the combination of explicit reasoning (traditional logic) and implicit pattern recognition (deep neural networks), especially as applied to reinforcement learning, natural language, and AI policy. See my interests and publications below.

I completed my master’s in computer science (ML specialization) as part of Georgia Tech’s OMSCS program.

Before this, I was a lawyer at Kirkland & Ellis in New York, where I worked on big corporate transactions (e.g., this and this). I also developed some interesting technology for corporate lawyers.

Before becoming a lawyer I was a fairly successful online poker player.

I received my J.D. in 2014 from Harvard Law School, where I was a fellow at the Olin Center for Law, Economics, and Business. My undergrad was in finance and economics at the Schulich School of Business in Toronto.

Research Interests

I’m interested in understanding and creating intelligence. For a focused selection of short term research questions I have, see here (1 page) or the August 2017 copy here (2 pages). Or see my recent papers. I also have an academic blog,, where I post small or incomplete ideas and tutorials.

Most recently, I’ve been working on enhancing reinforcement learning agents with source traces, a time-scale invariant model for probabalistic causation. Separately, I’ve been working toward developing an interpretable mechanism for reasoning over multiple representations. I see both avenues as critical for the development of artificial general intelligence, as any agent of general intelligence will need to answer the question “Why?”. This demands that the agent reason about causation; for humans, this typically involves generating multiple competing explanations, comparing their relative merits, and producing a “best” explanation along with a statement about its plausibility.


Rethinking the Discount Factor in Reinforcement Learning: A Decision Theoretic Approach

In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19). Honolulu, USA, 2019. (Paper, Slides, Poster, Bibtex)

Can all “rational” preference structures be represented using the standard RL model (the MDP)? This paper presents a minimal axiomatic framework for rationality in sequential decision making and shows that the implied cardinal utility function is of a more general form than the discounted additive utility function of an MDP. In particular, the developed framework allows for a state-action dependent “discount” factor that is not constrained to be less than 1 (so long as there is eventual long run discounting).

Challenging the MDP Status Quo: An Axiomatic Approach to Rationality for Reinforcement Learning Agents

Workshop Paper. The 1st Workshop on Goal Specifications for Reinforcement Learning, FAIM 2018. Stockholm, Sweden, 2018. (Paper, Poster)

This is the workshop version of the above full paper.

Source Traces for Temporal Difference Learning

In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18). New Orleans, USA, 2018. (Paper, Slides, Bibtex)

This paper develops source traces for reinforcement learning agents. Source traces provide agents with a causal model, and are related to both eligibility traces and the successor representation. They allow agents to propagate surprises (temporal differences) to known potential causes, which speeds up learning. One of the interesting things about source traces is that they are time-scale invariant, and could potentially be used to provide interpretable answers to questions of causality, such as “What is likely to cause X?”

I am currently working on extending this idea to continuous control with deep neural networks.

Reasoning for Reinforcement Learning

Workshop Paper. NIPS Hierarchical Reinforcement Learning Workshop. Long Beach, USA, 2017. (Paper, Poster)

This is a short abstract about some ideas I’m currently working on that connect implicit understanding (value functions) and explicit reasoning in the context of reinforcement learning. The idea is to create an architecture that is capable of integrating and simultaneously reasoning over multiple representations at different levels of abstraction.

Methods for Retrieving Alternative Contract Language Using a Prototype

In Proceedings of ICAIL ‘17: Sixteenth International Conference on Law and Artificial Intelligence. London, UK, 2017. Best student paper. (Paper, Slides, Bibtex)

This paper presents a search engine that finds similar language to a given query (the prototype) in a database of contracts. Results are clustered so as to maximize both coverage and diversity. This is useful for contract drafting and negotiation, administrative tasks and legal research.

An Alternative Arithmetic for Word Vector Analogies

Unpublished draft, dated June 7, 2016. (Paper)

This paper looks at word vector arithmetic of the type “king - man + woman = queen” and investigates treating the relationships between word vectors as rotations of the embedding space instead of as vector differences. This was a one week project of little practical significance, but with the advent of latent vector arithmetic (e.g., for GANs), it may be worth revisiting.

Punitive Damages in International Trade

Unpublished draft, dated April 22, 2014. (Paper)

What should the structure of rights, remedies, and enforcement look like in an efficient international trade agreement? In particular, do punitive damages have a place?

Designing Optimal Takeover Defenses

Unpublished draft, dated May 22, 2013. (Paper)

This paper analyzes the economic value of corporate takeover defenses, and argues for designing intermediate takeover defenses that balance (1) the interest of shareholders in management’s exploitation of insider information and (2) the entrenchment interest of management.

Examining Expected Utility Theory from Descriptive and Prescriptive Perspectives

Unpublished draft, dated January 2, 2010. (Paper)

This paper examines the history and validity of Expected Utility theory, with focus on its failures a descriptive model of human decisions. It is argued that the descriptive failures of Expected Utility lead to its incorrect usage as a prescriptive model, and a few brief examples of how one might properly construct a utility function are provided.

I’ve been interested in legal technology since my first law firm experience in the summer of 2013: the tech was underwhelming, and I spent hours on tasks that would take the right program seconds. This prompted me to write this paper (~6000 words) on potential tools for corporate lawyers and take my first formal programming class as an elective in my final year of law school.

During my time at Kirkland, I wrote a number of useful programs, which are summarized here (1 page) along with some other ideas I think would be useful. Here is some unsolicited praise for my legal software:

If it’s something you’re interested in, there is a decent opportunity for commercialization in this sector—I may be open to discussing. I haven’t pursued it due to my other interests (ironically, it was my desire to build smarter legal programs that led me to teach myself about machine learning and pursue my current non-law related research).