In this post, we will see what Apple MLX is, why it is interesting for people who have a Mac with Apple Silicon, how it is different from tools like Ollama and LM Studio, and how we can use it to run some local Large Language Models.
But first of all, what is MLX?
“MLX is an open-source framework created by Apple for Machine Learning on Apple Silicon.
In our case, we are not trying to train a neural network from zero or to create a complex Machine Learning algorithm. We just want to run a Large Language Model locally on our Mac.
But the point is that a Large Language Model is a Machine Learning model. When we use a model like Qwen, Gemma, DeepSeek or Llama, we are using a model that was already trained on a huge amount of data. When we ask a question, the model is not training again. It is doing inference.
Inference means that the model uses its trained weights to generate an answer.
So, even if from our point of view we are only writing a prompt and reading an answer, under the hood our Mac is running a Machine Learning workload.“
In a nutshell, MLX is the framework that allows these workloads to run efficiently on Apple Silicon.
Apple created MLX to work well with the Apple Silicon architecture and, especially, with unified memory.
Apple Silicon has a different architecture compared to many traditional computers. On many machines, CPU and GPU have separate memory, and data often needs to be copied from one place to another. On Apple Silicon, CPU and GPU can access the same memory pool. MLX was designed to take advantage of this model.
In my case, I will use a Mac Mini with Apple M4 and 24 GB of RAM.
The last point I want to higlight, is that we usually do not use MLX directly.
We use a Python package called ‘mlx-lm’, that it allows us to run and fine-tune Large Language Models on Apple Silicon.
MLX VS OLLAMA VS LM STUDIO
Before installing MLX, it is important to understand the difference between MLX, Ollama and LM Studio because they are all related to local AI, but they are not the same thing.
Ollama is probably the easiest way to run local Large Language Models.
We install it, we run a command like this:
ollama run qwen3
and Ollama downloads the model and runs it for us.
Ollama hides a lot of complexity. It manages the model, the runtime and the local API. This is very useful when we want to start quickly and we do not want to think too much about model formats, engines or Python environments.
LM Studio is different because it is a desktop application. It provides a graphical interface where we can search models, download them, load them and chat with them. It is very useful if we prefer to work with a UI instead of using only the terminal. LM Studio can also expose a local server compatible with the OpenAI API, so we can use a local model from our applications.
MLX is different from both.
MLX is not mainly a chat application and it is not mainly a desktop application.
MLX is the Apple framework used to run Machine Learning models efficiently on Apple Silicon.
Then, with mlx-lm, we can run LLMs from the terminal or from Python.
So, if we want the easiest experience, Ollama is probably the best choice.
If we want a graphical interface, LM Studio is probably the best choice.
If we are developers and we want to work directly with Python, Apple Silicon and MLX-compatible models, then MLX is very interesting.
GGUF MODELS VS MLX MODELS
When we start downloading local models, we often see different formats and this can be confusing.
One of the most common formats is GGUF.
GGUF is the format used by llama.cpp and it is widely used by tools like Ollama and LM Studio. A GGUF model is usually a single file, for example:
model-q4_k_m.gguf
This format is very practical because it contains the model weights and metadata in one file. It is also popular because it works on many platforms: macOS, Windows and Linux.
MLX models are different.
An MLX model is usually stored as a Hugging Face repository containing files like:
config.json
tokenizer.json
model.safetensors
or sometimes multiple safetensors files.
The model is converted or prepared to work efficiently with MLX and Apple Silicon.
So, the difference is not only the file extension. GGUF is mainly connected to the llama.cpp ecosystem. MLX models are connected to the Apple MLX ecosystem.
If we use Ollama or llama.cpp, usually we use GGUF models.
If we use mlx-lm, we use MLX-compatible models from Hugging Face.
This is important because we cannot simply take any GGUF model and run it directly with mlx-lm. We need an MLX-compatible model.
Before we continue, I’d like to spend just a minute to explain what Hugging Face is. Think of it as the “GitHub of the AI world”, a massive open-source platform where developers and researchers share machine learning models, datasets, and code. Instead of using a closed cloud API, it’s the place where we go to download actual model weights to run locally on our machine.
HOW TO INSTALL MLX-LM
Now, let’s install everything.
First of all, we create a folder for our tests:
mkdir mlx-tests
cd mlx-tests
Then, we create a Python virtual environment:
python3 -m venv .venv
Now, we activate it:
source .venv/bin/activate
Then, we upgrade pip:
pip install --upgrade pip
Finally, we install mlx-lm:
This command installs the package that allows us to run Large Language Models using Apple MLX.
In order to check if everything is installed correctly, we can run:
mlx_lm.generate -h
If we see the list of available options, it means that MLX-LM is installed correctly:
RUN THE FIRST MODEL
Now, we can try our first model.
For this post, we will use a Qwen model in MLX format:
mlx-community/Qwen3-4B-Instruct-2507-nvfp4
This is a good model to start with because it is not too big and it can run well on a Mac Mini M4 with 24 GB of RAM.
We can run it with this command:
mlx_lm generate \
--model mlx-community/Qwen3-4B-Instruct-2507-4bit \
--prompt "Explain what Apple MLX is in simple words." \
--max-tokens 300
The first time we run this command, the model will be downloaded from Hugging Face. After the download, MLX-LM will load the model and generate the response.
The important thing is that the model is running locally. We are not calling ChatGPT, we are not using an external API and we are not sending our prompt to a cloud service. We are using our Mac.
In order to check the model downloaded, we can use the command:
hf cache ls
In order to delete the models, we can use the command:
hf cache rm <model-id>
USING LM STUDIO WITH APPLE MLX
Until now, we used MLX from the terminal and from Python, but maybe we want something easier and more visual.
In this case, we can use LM Studio.
LM Studio is a desktop application that allows us to search, download and run local models using a graphical interface. The interesting thing is that LM Studio can also use MLX models on Apple Silicon.
We run LM Studio, we open the application and go to the model search section.
Here, we can filter by MLX:
When we find a model compatible with MLX, we can download it and load it directly inside LM Studio.
At that point, we can chat with the model using the LM Studio interface.
This is probably the easiest way to try MLX models with a UI because LM Studio manages the model and the runtime for us.
The most important thing to remember is that MLX is not just another local model runner. It is a Machine Learning framework optimized for Apple Silicon.
This makes it very interesting if we want to run local AI experiments directly on our Mac and if we want to integrate local models inside Python applications.
Obviously, Ollama and LM Studio remain very useful tools, especially when we want simplicity or a graphical interface.
But, if we are developers and we want more control, MLX is definitely something worth learning.