Using a Dockerized PI with Ollama to code Laravel project running in a Docker container. PI is a minimal harness, but it has no guardrails, so it can alter your system with the same rights as the user running it. So it can install packages. Dockerized PI image The Dockerfile: Create the image with: Not […]
Download a file from HugginfFace: This means: If you want to run this model with more context, you have to set the KV cache to a smaller size: This way it still runs on a Macbook Pro M1 with 32GB of RAM. Note that if you later want to run the same model, just use […]
There are different tokenizers for different LLM models. For example OpenAI uses tiktoken, while Mistral uses sentencepiece (or you can use AutoTokenizer from the transformers package). When use tokenizers: Tokenizer for Mistral and OpenAI See the difference between them. Use it with Docling when chunking