FMEngine

FMEngine is a utility library for training large foundation models. It is inspired by and built on top of many existing libraries and frameworks, such as DeepSpeed, torchtitan, torchtune.

Acknowledgement

FMEngine is primarily implemented and maintained by the Efficient Architecture and Systems Labs @ ETH Zurich.

We thank our friends for their generous support:

Contributors

@xzyaoi
@LorrinWWW
@fishiu: Scaling up Low Rank Adapters + Longer Contexts
@Taishi-N324: Benchmarking and Scaling Experiments
@chiennv2000: Mistral Integration