v0.4.0 - Executor and ChatSession

Version 0.4.0 introduces many break changes. However we strongly recommend to upgrade to 0.4.0 because it provides better abstractions and stability by refactoring the framework. The backend v0.3.0 and v0.3.1 still works for LLamaSharp v0.4.0.

The main changes:

1. Add three-level abstractions: LLamaModel, LLamaExecutor and ChatSession.
2. Fix the BUG of saving and loading state.
3. Support saving/loading chat session directly.
4. Add more flexible APIs in the chat session.
5. Add detailed documentations: https://scisharp.github.io/LLamaSharp/0.4/

Acknowledge

During the development, thanks a lot for the help from @TheTerrasque ! His/Her fork gives us many inspirations. Besides, many thanks for the following contributors!

• MacOS Arm64 support by @SignalRT in #24
• Fixed a typo in FixedSizeQueue by @mlof in #25
• Document interfaces by @mlof in #26

New Contributors

• @SignalRT made their first contribution in #24
• @mlof made their first contribution in #25

v0.3.0 - Load and save state

1. Support loading and saving state.
2. Support tokenization and detokenization.
3. Fix BUGs of instruct mode.
4. break change: n_parts param is removed.
5. break change: LLamaModelV1 is dropped.
6. Remove dependencies for third-party loggers.
7. Verified model repo is added on huggingface.
8. Optimize the examples.

v0.2.3 - Inference BUG Fix

Fix some strange behaviors of model inference.

v0.2.2 - Embedder

1. Sync with the latest llama.cpp master branch.
2. Add LLamaEmbedder to support to get the embeddings only.
3. Add n_gpu_layers and prompt_cache_all param.
4. Split the package into main package + backend package.

v0.2.1 - Chat session, quantization and Web API

1. Add basic APIs and chat session.
2. Support quantization.
3. Add Web API support.