It’s been a while since you heard from me. I joined Google DeepMind two months ago, and the weeks have been extremely busy and packed with exciting developments.

Below is a quick summary of everything that happened. As part of the Developer Relations Engineering team for Gemini and Gemma, I've had a front-row seat to most of those launches:

Model updates:

• Gemini 2.5 Pro

• Gemini 2.0 Flash native image generation

• Gemini Robotics

• Gemini Embedding

• Imagen 3

• Gemini 2.0 Flash

• Gemini 2.0 Flash-Lite

• Gemini 2.0 Pro

• Gemma 3

• ShieldGemma 2

• PaliGemma 2 Mix

Product improvements:

• YouTube link support

• Conversation branching in Google AI Studio

• AI co-scientist

• Audio overview & Canvas mode (Gemini app)

• AI Mode powered by Gemini (Google Search)

• Code execution as a tool

Dev Ex:

• Gen AI Python SDK v1.0.0

• Gen AI JS/TS SDK (preview launch)

• Improved OpenAI SDK compatibility layer

• Major docs updates (still ongoing)

• New cookbooks

Hands-on tutorials from me (not official Google tutorials):

• Get started with Gemini's multimodal capabilities

• Image creation and editing with Gemini 2.0

• Create workout summaries with Gemma 3 1B

The pace here is incredible, and moving forward, I plan to share more updates and insights into the exciting work I see at Google DeepMind (but note that I can't talk about unreleased products).

I know that my newsletter hasn't been very consistent, and with this new content shift, I completely understand if you decide to unsubscribe. Thank you all for your past engagement and support! And for those of you who will be staying, my goal is to make this new focus on AI research, product updates, and tutorials genuinely interesting and valuable.

Best,

Patrick

It’s been a while, but now I’m back with a cool project and tutorial: Learn how to build an AI-powered locally running typing assistant with Ollama and Mistral 7B.

In this post, I'll show you how to do it.

It's a script with less than 100 lines of code that can run in the background and listen to hotkeys, then uses a Large Language Model to fix the text.

It’s inspired by this tweet:

"GPT is surprisingly good at correcting minor typos, so you can write really really fast, ignore mistakes and keep going, and it comes out just fine." - Andrej Karpathy

The full code is on GitHub: https://github.com/patrickloeber/ai-typing-assistant

And here's a demo and a step-by-step coding tutorial if you prefer video content:

Project setup:

Ollama Installation: https://github.com/ollama/ollama. Then run

ollama run mistral:7b-instruct-v0.2-q4_K_S.

I found that Mistal 7B Instruct (quantized) works well for this task, but feel free to try other models, too :)

It relies on those dependencies:

- pynput: To listen to hotkeys and control the keyboard

- pyperclip: Clipboard functions to copy and paste text

- httpx: To send a POST request to Ollama running on your localhost

Install the dependencies with pip:

pip install pynput pyperclip httpx

Everything should work cross-platform, only the shortcuts need to be adapted for your system.

Hotkey listener in Python

This is how you can write a Hotkey listener with Python and pynput:

from pynput import keyboard

def on_f9():

    fix_current_line()


def on_f10():

    fix_selection()


with keyboard.GlobalHotKeys({"<101>": on_f9, "<109>": on_f10}) as h:

    h.join()

This runs endlessly in a separate thread until you stop the script.

Modify lines and selected text with Python

To modify selected lines, we only need the functionality to copy and paste text (= Cmd+C and Cmd+V), and to access and modify the clipboard with Python. For this, set up a keyboard controller with pynput, and use pyperclip for the clipboard functions:

import time

from pynput.keyboard import Key, Controller

import pyperclip


controller = Controller()


def fix_current_line():

    # macOS short cut to select current line: Cmd+Shift+Left

    controller.press(Key.cmd)

    controller.press(Key.shift)

    controller.press(Key.left)

    controller.release(Key.cmd)

    controller.release(Key.shift)

    controller.release(Key.left)


    fix_selection()


def fix_selection():

    # 1. Copy selection to clipboard

    with controller.pressed(Key.cmd):

        controller.tap("c")


    # 2. Get the clipboard string

    time.sleep(0.1)

    text = pyperclip.paste()


    # 3. Fix string

    if not text:

        return

    fixed_text = fix_text(text)

    if not fixed_text:

        return


    # 4. Paste the fixed string to the clipboard

    pyperclip.copy(fixed_text)

    time.sleep(0.1)


    # 5. Paste the clipboard and replace the selected text

    with controller.pressed(Key.cmd):

        controller.tap("v")

Use Ollama and Mistral 7B to fix text

Ollama comes with a REST API that's running on your localhost out of the box. So after the installation and downloading of the model, we only need to implement logic to send a POST request.

We also use a simple prompt that tells the model to fix all typos, casing, and punctuation:

import httpx

OLLAMA_ENDPOINT = "http://localhost:11434/api/generate"

OLLAMA_CONFIG = {

    "model": "mistral:7b-instruct-v0.2-q4_K_S",

    "keep_alive": "5m",

    "stream": False,

}


PROMPT_TEMPLATE = Template(

    """Fix all typos and casing and punctuation in this text, but preserve all new line characters:

$text

Return only the corrected text, don't include a preamble.

"""

)


def fix_text(text):

    prompt = PROMPT_TEMPLATE.substitute(text=text)

    response = httpx.post(

        OLLAMA_ENDPOINT,

        json={"prompt": prompt, **OLLAMA_CONFIG},

        headers={"Content-Type": "application/json"},

        timeout=10,

    )

    if response.status_code != 200:

        print("Error", response.status_code)

        return None

    return response.json()["response"].strip()

Run the typing assistant

Put everything together and start the assistant:

python main.py

Hotkeys you can then press:

- F9: Fixes the current line (without having to select the text)

- F10: Fixes the current selection

Note: The code works on macOS. The underlying shortcuts in the code like Cmd+Shift+Left, Cmd+C, Cmd+V might have to be changed for Linux & Windows (e.g. Key.cmd -> Key.ctrl).

Customize

The hotkeys, prompt, and Ollama config can be easily customized and extended in the code.

You can easily add hotkeys that summarize or generate new text.

For example, here are some fun other prompt templates you can try:

from string import Template


PROMPT_TEMPLATE_FIX_TEXT = Template(

    """Fix all typos and casing and punctuation in this text, but preserve all new line characters:

$text

Return only the corrected text."""

)


PROMPT_TEMPLATE_GENERATE_TEXT = Template(

    """Generate a snarky paragraph with 3 sentences about the following topic:

$text

Return only the corrected text."""

)


PROMPT_TEMPLATE_SUMMARIZE_TEXT = Template(

    """Summarize the following text in 3 sentences:

$text

Return only the corrected text."""

)

Have fun with it!

Welcome to another edition of my monthly newsletter. Let’s dive right in:

❓What can you expect?

This newsletter features tutorials, news, cool repos, and educational nuggets related to Python & AI.

⚡️Sponsor

Build a software business faster in pure Python with PySaaS. PySaaS a boilerplate Python codebase that takes care of the fundamental structure and components standard to all SaaS applications such as accepting payments, authentication, an integrated blog, and a nice landing page.

📺📚Tutorials

Here’s a short selection of interesting tutorials from the last weeks that are worth watching:

• Harvard CS50’s Introduction to Programming with Python is now available for free on freeCodeCamp’ YouTube channel.

• Master PDF Chat with LangChain from Sam Witteveen.

• Vector Databases simply explained in 5 minutes. A video I created for the AssemblyAI channel.

• The ultimate introduction to modern GUIs in Python with tkinter. An extensive 18h course from Clear Code.

• DeepLearning.AI launched 3 new courses that are available for free for a limited time: LangChain for LLM Development, How Diffusion Models Work, and Building Systems with the ChatGPT API

• Article: How to split a List into equally sized chunks in Python. Python 3.12 will get a cool new method to do this for you!

🐍 News from the Python Ecosystem

• Chris Lattner announced a new programming language called Mojo🔥, which builds on top of Python and is designed specifically for AI applications. They claim it’s up to 35000 times faster than Python. I made a video about it here.

• PyPI was subpoenaed. They published a blog post about it here for transparency. They also announced that every account that maintains any project or organization on PyPI will be required to enable 2FA on their account by the end of 2023.

• In April, the pip team released Pip version 23.1 which solves dependencies much faster thanks to an improved backtracking algorithm. I recommend upgrading to the newest version with python -m pip install --upgrade pip.

• The open source library LlamaIndex raises $8.5M seed round to build out the data framework for LLMs.

🤝Cool Repos

• lit-parrot: Hackable implementation of state-of-the-art open-source large language models such as StableLM and Falcon.

• mlc-llm: a universal solution that allows any language models to be deployed natively on a diverse set of hardware backends and native applications.

• MiniGPT-4: Enhancing Vision-language Understanding with Advanced Large Language Models

• starcoder: StarCoder is a language model (LM) trained on source code and natural language text

• assemblyai-python-sdk: Finally an SDK from AssemblyAI that makes it super simple to transcribe and analyze audio files.

💡Python Tip

The print function can take additional arguments such as sep, end, and file:

# 1. sep defines the separator between all objects, default is a space

print("How", "are", "you", sep="-")
# How-are-you

# 2. end defines the character in the end, default is a new-line char

print("Hello", end="")
print("World")
# HelloWorld

# 3. file to write the output into a file instead of stdout

f = open("test.txt", "a")
print("This goes into a file", file=f)
f.close()

I hope you enjoyed the read!

Patrick

Welcome to another edition of my monthly newsletter. Let’s dive right in:

❓What can you expect?

This newsletter features tutorials, news, cool repos, and educational nuggets related to Python & AI.

📺📚Tutorials

Here’s a short selection of interesting tutorials from last month that are worth watching. The first three videos are from me:

• LangChain Crash Course - Build apps with language models. You can also find a written guide here.

• PyScript is officially here! Build web apps with Python & HTML

• Auto-GPT, BabyAGI, and 4 autonomous agents you must try

• 5 Steps to Build a Question Answering PDF Chatbot

• Complete Docker Course - From BEGINNER to PRO!

🐍 News from the Python Ecosystem

• The Python framework LangChain announced their $10M seed round

• PEP 684 has been accepted which introduces a per-interpreter GIL in Python 3.12

• Pandas 2.0 was released

• Anaconda Launches PyScript.com, a new platform to build web apps with Python and HTML

🤝Cool Repos

• Auto-GPT: Do I need to say something about this repo? 120K stars in 4 weeks🤯

• babyagi: An AI-powered task management system to solve tasks autonomously with the help of GPT-4. The code is intentionally kept minimal, so exploring the code is nice.

• llama-index: A project that provides a central interface to connect your LLM's with external data.

• weaviate: An open source vector database that stores both objects and vectors, allowing for combining vector search with structured filtering.

• lit-llama: An independent implementation of LLaMA based on nanoGPT that is fully open source under the Apache 2.0 license.

💡Python Tip

Do you know about the built-in any() function? any() returns True if any element of the iterable is true. Look at this example and how it can be used to refactor code:

numbers = [-1, -2, -4, 0, 3, -7]

# manually looping
has_positives = False
for n in numbers:
    if n > 0:
        has_positives = True
        break

# refactor using any()✅
has_positives = any(n > 0 for n in numbers)

I hope you enjoyed the read!

Patrick

Welcome to another edition of my monthly newsletter. Let’s dive right in:

❓What can you expect?

This newsletter features tutorials, news, cool repos, and educational nuggets related to Python & AI.

📺📚Tutorials

Here’s a short selection of interesting tutorials from last month that are worth watching:

• Python for AI developers: A free course on YouTube from me and my colleague Mısra that helps you to build with AI

• GPT-4 Developer Livestream from OpenAI

• Generative AI and LLMs - Playlist from James Briggs

• Harvard CS50’s Web Programming with Python and JavaScript – Full University Course available on freeCodeCamp’s YouTube channel

🐍 News from the Python Ecosystem

• PyTorch 2.0 is officially here. See the announcement.

• PyTorch Lightning 2.0 is here. Among other changes, the team is introducing Fabric, a fast and lightweight way to scale PyTorch models with just a few lines of code.

• Python 3.12.0 alpha 6 released - In case you missed it, you can already play around with the next Python version.

• Pandas 2.0 is almost here! Keep an eye out in the coming days. See the What’s New blog post and watch the releases page.

🤖AI Month

This month was absolutely crazy in the AI space. I summarized the most important releases in this tweet:

I won’t cover all the details in this newsletter, but if you want to follow a newsletter that is focused on AI only, I created a list of 15 great AI newsletters here.

🤝Cool Repos

• python-github-action-template: A template that lets you easily schedule a Python script with GitHub Actions

• langchain: LangChain is a framework for developing applications powered by language models. It facilitates building applications with LLMs through composability.

💡Python Tip

Use more dataclasses in your Python code. Gets rid of so much boilerplate:

from dataclasses import dataclass

@dataclass
class Student:
    age: int
    name: str

student = Student(22, "Anna")
print(student)
# Student(age=22, name='Anna')

I hope you enjoyed the read!

Patrick

This is Patrick writing to you from a new platform. You might be surprised to see a new email address since you probably subscribed on the Python Engineer website or on Revue. I moved my newsletter to Substack, so don’t worry, it’s still me.

❓What can you expect?

This will be a newsletter featuring tutorials, news, trends, and educational nuggets related to Python & AI. For the time being, I want to send out new issues every month.

⚡️Thanks to our Sponsors who keep this Newsletter free

This first issue is brought to you by Underfitted, another AI newsletter from my friend Santiago. Every Friday, you'll get a short, exciting story about Artificial Intelligence that will inspire and make you smarter. And all of this in fewer than 500 words! Click here to subscribe.

📺📚Tutorials

Tutorial from the past weeks you might have missed:

• How to create a great dev environment with Docker

• How to automate Notion with Python | Notion API Tutorial

Not from me but really awesome:

• Andrej Karpathy: Let's build GPT from scratch

• freeCodeCamp: AlphaZero from Scratch

🐍 News from the Python Ecosystem

• Pydantic, the popular Open-Source data validation framework is starting a company

• A new solution for packaging in Python? Python Packaging Strategy Discussion

🤝Cool Repos

• picoGPT: An unnecessarily tiny implementation of GPT-2 in NumPy

• paper-qa: question and answering from PDFs or text files

💡Python Tip

How to pad zeros to a String: zfill, rjust, or String formatting:

"42".zfill(5)         # '00042'
"-42".zfill(5)        #'-0042'

"42".rjust(5)         # '   42'
"42".rjust(5, "0")    # '00042'
"-42".rjust(5, "0")   # '00-42'

n = 42
f'{n:05d}'            # '00042'
n = -42
f'{n:05d}'            #'-0042'

I hope you enjoyed the read!

Patrick

Subscribe now