AI NEWS.ccstable-code-3b
`stable-code-3b` is a 2.7B billion parameter decoder-only language model pre-trained on 1.3 trillion tokens of diverse textual and code datasets. `stable-code-3b` is trained on 18 programming languages (selected based on the 2023 StackOverflow Developer Survey) and demonstrates state-of-the-art performance (compared to models of similar size) on the MultiPL-E metrics across multiple programming languages tested using BigCode's Evaluation Harness. **Key Features** * Fill in Middle Capability (FIM) * Supports Long Context, trained with Sequences upto 16,384 The model is a decoder-only transformer similar to the LLaMA (Touvron et al., 2023) architecture with the following modifications:
`stable-code-3b` is a 2.7B billion parameter decoder-only language model pre-trained on 1.3 trillion tokens of diverse textual and code datasets. `stable-code-3b` is trained on 18 programming languages (selected based on the 2023 StackOverflow Developer Survey) and demonstrates state-of-the-art performance (compared to models of similar size) on the MultiPL-E metrics across multiple programming languages tested using BigCode's Evaluation Harness. **Key Features** * Fill in Middle Capability (FIM) * Supports Long Context, trained with Sequences upto 16,384 The model is a decoder-only transformer similar to the LLaMA (Touvron et al., 2023) architecture with the following modifications: