santacoder

Model overview

The SantaCoder models are a series of 1.1B parameter models trained on the Python, Java, and JavaScript subset of The Stack (v1.1) (which excluded opt-out requests). The main model uses Multi Query Attention, a context window of 2048 tokens, and was trained using near-deduplication and comment-to-code ratio as filtering criteria and using the Fill-in-the-Middle objective. In addition there are several models that were trained on datasets with different filter parameters and with architecture and objective variations. The `final` model is the best performing model and was trained twice as long (236B tokens) as the others. This checkpoint is the default model and available on the `main` branch. All other checkpoints are on separate branches with according names. The model was trained on GitHub code. As such it is _not_ an instruction model and commands like "Write a function that computes the square root." do not work well. You should phrase commands like they occur in source code such as comments (e.g. `# the following function computes the sqrt`) or write a function signature and docstring and let the model complete the function body.

Key features

特点摘要待依据来源补齐，不凭模型名称推断能力。

Limitations & considerations

Sources are insufficient to establish specific limitations. Verify task performance, latency, data policies, licensing and version lifecycle.

Source prices & cost analysis

Prices vary by channel, region and configuration. Channel quotes are listed independently and are not treated as a single official provider price.

Public benchmarks & run configuration

暂无已采集的可核验公开评分。模型未上榜不代表能力较低。

Frequently asked questions

Sources & traceability

许可证：bigcode-openrail-m。模型介绍与第三方评测均以原来源的具体版本和方法为准。