Lilian Weng(@lilianweng)

Inkling is our open weights model. It aims to serve as a foundation with solid performance across ...

Lilian Weng (@lilianweng) — Wed, 15 Jul 2026 19:14:33 GMT

Inkling is our open weights model. 

It aims to serve as a foundation with solid performance across a broad categories of capabilities, for use in practice and customization.

Play it on Tinker! 😄

Thinking Machines@thinkymachines

Today, we are introducing Inkling.

Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available.

thinkingmachines.ai/news/introduci…

Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵

🔗 View Quoted Tweet

💬37🔄51❤️1403👀89603📊140

⚡ Powered by xgo.ing

new post on harness engineering for AI self-improvement: https://t.co/ZYvGfVs61k It is hard to fore...

Lilian Weng (@lilianweng) — Tue, 07 Jul 2026 05:58:07 GMT

new post on harness engineering for AI self-improvement: lilianweng.github.io/posts/2026-07-…

It is hard to forecast how much the future of RSI will rely on harnesses. Likely harness engineering will evolve in the direction of self-improvement and enable auto-research, and, in turn, smarter models keeps harnesses simple. 

Even when many harness improvement get eventually internalized into core model, the need to specify goals and context will not disappear.

💬110🔄770❤️5269👀768552📊1774

⚡ Powered by xgo.ing

Soon I should set up the model to update Lil’Log automatically but I’m not there yet.

Lilian Weng (@lilianweng) — Thu, 25 Jun 2026 20:18:34 GMT

Soon I should set up the model to update Lil’Log automatically but I’m not there yet.

💬7🔄2❤️86👀26364📊15

⚡ Powered by xgo.ing

I wonder how many more people will just ask model to summarize the post, in comparison to a year ago...

Lilian Weng (@lilianweng) — Thu, 25 Jun 2026 20:17:01 GMT

I wonder how many more people will just ask model to summarize the post, in comparison to a year ago. 😂

💬16🔄0❤️155👀34816📊27

⚡ Powered by xgo.ing

A super long overdue (3+ years?) post on scaling laws. Compute is expensive. Scaling laws are a way...

Lilian Weng (@lilianweng) — Thu, 25 Jun 2026 20:06:39 GMT

A super long overdue (3+ years?) post on scaling laws.

Compute is expensive. Scaling laws are a way to help us reason about the optimal compute allocation between data and model size before committing to a large run. 

The post covers what scaling laws predict, how compute-optimal allocation works, why Kaplan et al. and Chinchilla disagree, and how data limits + fitting details make extrapolation tricky.

lilianweng.github.io/posts/2026-06-…

💬67🔄586❤️4677👀443564📊1417

⚡ Powered by xgo.ing

We would love to see more collaboration and research in the field of human-AI interactivity. Check i...

Lilian Weng (@lilianweng) — Tue, 19 May 2026 17:40:58 GMT

We would love to see more collaboration and research in the field of human-AI interactivity. Check it out!

Thinking Machines@thinkymachines

We are offering grants of $100,000 + Tinker credits to researchers advancing the field of human-AI interactivity. Submit your proposals by June 19th!

thinkingmachines.ai/news/interacti…

🔗 View Quoted Tweet

💬28🔄23❤️445👀121916📊69

⚡ Powered by xgo.ing

I only recently read more about the concept of system accidents by Charles Perrow, very insightful a...

Lilian Weng (@lilianweng) — Mon, 18 May 2026 00:58:07 GMT

I only recently read more about the concept of system accidents by Charles Perrow, very insightful and relatable.

💬30🔄12❤️447👀90317📊106

⚡ Powered by xgo.ing

In the past few months, we had a lot of fun (and stress 😅) to produce 12 versions (+ many subversio...

Lilian Weng (@lilianweng) — Mon, 11 May 2026 20:50:27 GMT

In the past few months, we had a lot of fun (and stress 😅) to produce 12 versions (+ many subversions) and 137 pages in our training run log book.

Turns out human-human collaboration is important to improving human-AI collaboration. 😊

💬71🔄50❤️1000👀232591📊202

⚡ Powered by xgo.ing

Building technologies for better human-AI collaboration on next gen hardware at scale. Exciting.

Lilian Weng (@lilianweng) — Tue, 10 Mar 2026 17:08:44 GMT

Building technologies for better human-AI collaboration on next gen hardware at scale. Exciting.

Thinking Machines@thinkymachines

We are partnering with @nvidia to power our frontier model training and platforms delivering customizable AI.

thinkingmachines.ai/news/nvidia-pa…

🔗 View Quoted Tweet

💬32🔄20❤️481👀96166📊61

⚡ Powered by xgo.ing

I’ve been telling people this a lot today: I enjoy so much working with people who care about what t...

Lilian Weng (@lilianweng) — Thu, 15 Jan 2026 04:03:24 GMT

I’ve been telling people this a lot today: I enjoy so much working with people who care about what they are building and craftsmanship. It is a privilege to have a chance to work on something I’m passionate about, beyond making a living. I cherish it and don’t take it for granted.

💬69🔄67❤️1705👀205478📊248

⚡ Powered by xgo.ing

On-policy distillation provides an elegant way to use the teacher model as a process reward model to...

Lilian Weng (@lilianweng) — Mon, 27 Oct 2025 17:31:45 GMT

On-policy distillation provides an elegant way to use the teacher model as a process reward model to provide dense reward while preventing SFT style "OOD shock" during rollout.

Thinking Machines@thinkymachines

Our latest post explores on-policy distillation, a training approach that unites the error-correcting relevance of RL with the reward density of SFT. When training it for math reasoning and as an internal chat assistant, we find that on-policy distillation can outperform other approaches for a fraction of the cost.

thinkingmachines.ai/blog/on-policy…

🔗 View Quoted Tweet

💬31🔄51❤️779👀162872📊118

⚡ Powered by xgo.ing

GPUs are expensive and setting up the infrastructure to make GPUs work for you properly is complex, ...

Lilian Weng (@lilianweng) — Wed, 01 Oct 2025 18:29:27 GMT

GPUs are expensive and setting up the infrastructure to make GPUs work for you properly is complex, making experimentation on cutting-edge models challenging for researchers and ML practitioners. 

Providing high quality research tooling is one of the most effective ways to improve research productivity of the wider community and Tinker API is one step towards our mission there.

Tinker API is built on top of our experimental results on fine-tuning with LoRA: thinkingmachines.ai/blog/lora/

Beta starts and you can join the waitlist today: thinkingmachines.ai/tinker/

💬48🔄130❤️2082👀214796📊355

⚡ Powered by xgo.ing

Looking through those little hidden gem stories in the footnote, you will find it so inspiring that ...

Lilian Weng (@lilianweng) — Fri, 26 Sep 2025 19:03:49 GMT

Looking through those little hidden gem stories in the footnote, you will find it so inspiring that researchers with interests on the same topic are able to work together to advance a field despite their roles and locations. This is the power of open science and community.

Thinking Machines@thinkymachines

Efficient training of neural networks is difficult. Our second Connectionism post introduces Modular Manifolds, a theoretical step toward more stable and performant training by co-designing neural net optimizers with manifold constraints on weight matrices. 

thinkingmachines.ai/blog/modular-m…

We explore a fundamental understanding of the geometry of neural network optimization.

🔗 View Quoted Tweet

💬16🔄43❤️772👀131799📊94

⚡ Powered by xgo.ing

To be clear, here the (first) Thinking Machines Corporation (https://t.co/DTcUwSNI6C) I referred to ...

Lilian Weng (@lilianweng) — Thu, 11 Sep 2025 02:30:29 GMT

To be clear, here the (first) Thinking Machines Corporation (en.m.wikipedia.org/wiki/Thinking_…) I referred to was founded in 1983 :)

💬6🔄3❤️46👀31609📊11

⚡ Powered by xgo.ing

Besides the fun fact that Connectionism is connected with the early days of the AI field and highlig...

Lilian Weng (@lilianweng) — Wed, 10 Sep 2025 17:24:30 GMT

Besides the fun fact that Connectionism is connected with the early days of the AI field and highlights similarities between neural networks and human brains, the flagship product of the (first) Thinking Machines is named Connection Machine. — 🧑‍🎓Enjoy reading and more is coming!

Thinking Machines@thinkymachines

Today Thinking Machines Lab is launching our research blog, Connectionism. Our first blog post is “Defeating Nondeterminism in LLM Inference”

We believe that science is better when shared. Connectionism will cover topics as varied as our research is: from kernel numerics to prompt engineering. Here we share what we are working on and connect with the research community frequently and openly.

The name Connectionism is a throwback to an earlier era of AI; it was the name of the subfield in the 1980s that studied neural networks and their similarity to biological brains.

thinkingmachines.ai/blog/defeating…

🔗 View Quoted Tweet

💬29🔄71❤️1311👀184554📊169

⚡ Powered by xgo.ing

Welcome @_kevinlu to the team! Exciting to work together again.

Lilian Weng (@lilianweng) — Mon, 18 Aug 2025 19:49:47 GMT

Welcome @_kevinlu to the team! Exciting to work together again.

Kevin Lu@_kevinlu

I recently joined @thinkymachines -- super excited to work with the team, I think we have the highest density of research talent in the world 🙂

we have a very ambitious roadmap ahead, the right team to work on it, & I think now is a great time to join; you should reach out to the team if that excites you!

🔗 View Quoted Tweet

💬10🔄6❤️307👀90295📊31

⚡ Powered by xgo.ing

Yes - 🥳 Thinky starts hiring again: https://t.co/Dy97oJHbxy

Lilian Weng (@lilianweng) — Tue, 15 Jul 2025 18:11:24 GMT

Yes - 🥳 Thinky starts hiring again:thinkingmachines.paperform.coy

Mira Murati@miramurati

Thinking Machines Lab exists to empower humanity through advancing collaborative general intelligence.

We're building multimodal AI that works with how you naturally interact with the world - through conversation, through sight, through the messy way we collaborate. We're excited that in the next couple months we’ll be able to share our first product, which will include a significant open source component and be useful for researchers and startups developing custom models. Soon, we’ll also share our best science to help the research community better understand frontier AI systems.

To accelerate our progress, we’re happy to confirm that we’ve raised $2B led by a16z with participation from NVIDIA, Accel, ServiceNow, CISCO, AMD, Jane Street and more who share our mission.

We’re always looking for extraordinary talent that learns by doing, turning research into useful things. We believe AI should serve as an extension of individual agency and, in the spirit of freedom, be distributed as widely and equitably as possible.  We hope this vision resonates with those who share our commitment to advancing the field. If so, join us. thinkingmachines.paperform.co

🔗 View Quoted Tweet

💬20🔄24❤️574👀93733📊78

⚡ Powered by xgo.ing

I still find it mysterious whether and how intelligence and capabilities transfer between domains an...

Lilian Weng (@lilianweng) — Sun, 13 Jul 2025 21:19:46 GMT

I still find it mysterious whether and how intelligence and capabilities transfer between domains and skills - from meta learning during early days to more recent question like whether solving maths helps writing a good essay. 

Sometime I feel a bit pessimistic given not enough evidence I’ve seen. Would like to get more suggestions and pointers to papers on this topic of generalization in the thread! 🧵

💬61🔄41❤️778👀111815📊198

⚡ Powered by xgo.ing

Probably the first product Thinky will build is a full panel of dials that researchers can use to ph...

Lilian Weng (@lilianweng) — Sun, 25 May 2025 04:08:30 GMT

Probably the first product Thinky will build is a full panel of dials that researchers can use to physically adjust all the hparams during training. We gonna do hardware one day and it is the time 😂

Stephen Roller@stephenroller

Some teams use sweeps, heuristics, or scaling laws to determine their training LR. At Character, we just have Noam Shazeer dial it to the right value.

Your browser does not support the video tag.

🔗 View on Twitter

🔗 View Quoted Tweet

💬30🔄18❤️487👀131367📊72

⚡ Powered by xgo.ing

Giving your models more time to think before prediction, like via smart decoding, chain-of-thoughts ...

Lilian Weng (@lilianweng) — Sat, 17 May 2025 15:09:35 GMT

Giving your models more time to think before prediction, like via smart decoding, chain-of-thoughts reasoning, latent thoughts, etc, turns out to be quite effective for unblocking the next level of intelligence.

New post is here :) 

“Why we think”: lilianweng.github.io/posts/2025-05-…

💬141🔄481❤️3607👀403989📊928

⚡ Powered by xgo.ing

When a new dataset comes out, I get excited and check it out and then only realize that this is anot...

Lilian Weng (@lilianweng) — Mon, 12 May 2025 20:51:11 GMT

When a new dataset comes out, I get excited and check it out and then only realize that this is another meta-mixed dataset combining a collections of other existing datasets. My brain immediately acts like "oh fork ... contamination!" No meta-meta-mixed dataset plzzzz :lolsob:

💬29🔄35❤️610👀86362📊100

⚡ Powered by xgo.ing

Nope what’s that?

Lilian Weng (@lilianweng) — Tue, 29 Apr 2025 09:01:15 GMT

Nope what’s that?

Isa Fulford@isafulf

me at the iclr openai recruiting event: 

random man:
have you heard of arxiv?

🔗 View Quoted Tweet

💬17🔄9❤️243👀84077📊31

⚡ Powered by xgo.ing

See you at #ICLR2025 soon. Excited about chatting with many of you about Thinking Machines and what ...

Lilian Weng (@lilianweng) — Mon, 14 Apr 2025 18:29:07 GMT

See you at #ICLR2025 soon. Excited about chatting with many of you about Thinking Machines and what we have been up to!

Thinking Machines@thinkymachines

Thinking Machines is hosting a happy hour in Singapore during #ICLR2025 on Friday, April 25: lu.ma/ecgmuhmx

Come eat, drink, and learn more about us!

🔗 View Quoted Tweet

💬11🔄11❤️245👀67980📊29

⚡ Powered by xgo.ing

👩‍🍳Actively cooking the next blog post. Tiny teaser: It is spiritually related to our new company...

Lilian Weng (@lilianweng) — Sat, 05 Apr 2025 21:39:43 GMT

👩‍🍳Actively cooking the next blog post.

Tiny teaser: It is spiritually related to our new company.

💬18🔄15❤️609👀66047📊78

⚡ Powered by xgo.ing

Rule-based rewards (RBRs) use model to provide RL signals based on a set of safety rubrics, making i...

Lilian Weng (@lilianweng) — Wed, 24 Jul 2024 17:30:22 GMT

Rule-based rewards (RBRs) use model to provide RL signals based on a set of safety rubrics, making it easier to adapt to changing safety policies wo/ heavy dependency on human data. It also enables us to look at safety and capability in a more unified lens as a more capable grader model gives us higher quality RL signals.

💬12🔄43❤️307👀143783📊65

⚡ Powered by xgo.ing

Rebirth

Lilian Weng (@lilianweng) — Wed, 22 Nov 2023 06:28:48 GMT

Rebirth

OpenAI@OpenAI

We have reached an agreement in principle for Sam Altman to return to OpenAI as CEO with a new initial board of Bret Taylor (Chair), Larry Summers, and Adam D'Angelo.

We are collaborating to figure out the details. Thank you so much for your patience through this.

🔗 View Quoted Tweet

💬20🔄11❤️588👀120332📊55

⚡ Powered by xgo.ing

About 650 / 770 signed at this moment. As people start waking up, more will come. All the efforts st...

Lilian Weng (@lilianweng) — Mon, 20 Nov 2023 16:12:55 GMT

About 650 / 770 signed at this moment. As people start waking up, more will come. All the efforts started after 1:30 AM, 500+ within two hours and all of this after 2 crazy days with very little sleep.

💬145🔄538❤️4858👀2610649📊810

⚡ Powered by xgo.ing

(1/3) Alongside Superalignment team, my team is working on the practical side of alignment: Building...

Lilian Weng (@lilianweng) — Fri, 07 Jul 2023 03:35:11 GMT

(1/3) Alongside Superalignment team, my team is working on the practical side of alignment: Building systems to enable safe AI deployment. We are looking for strong research engineers and scientists to join the efforts.

💬18🔄48❤️401👀382087📊58

⚡ Powered by xgo.ing

Check out this post with @gdb, if you are curious about how to train large deep learning models and ...

Lilian Weng (@lilianweng) — Thu, 09 Jun 2022 17:49:33 GMT

Check out this post with @gdb, if you are curious about how to train large deep learning models and you may find it is easier than you expected :) Also we are hiring!

OpenAI@OpenAI

Techniques for training large neural networks, by @lilianweng and @gdb: openai.com/blog/technique…

🔗 View Quoted Tweet

💬7🔄27❤️239📊35