A project on Github that offers to run LLM on AMD graphics gas pedals using the Docker container. The image is designed to work with Hugging Face models, primarily the LLama family of models.

To run it, you need to have an AMD GPU with ROCm support (version 5.4.2 or higher) and Docker installed.

To adapt the inference logic to your needs, make appropriate changes to the run_inference.py file followed by rebuilding the Docker image.

The project provides an Aptfile file containing a list of required ROCm packages (rocm-dev, rocm-libs, rocm-cmake, miopen-hip and rocblas) to be installed in the Docker container.

A project on Github that offers to run LLM on AMD graphics gas pedals using the Docker container. The image is designed to work with Hugging Face models, primarily the LLama family of models.

To run it, you need to have an AMD GPU with ROCm support (version 5.4.2 or higher) and Docker installed.

To adapt the inference logic to your needs, make appropriate changes to the run_inference.py file followed by rebuilding the Docker image.

The project provides an Aptfile file containing a list of required ROCm packages (rocm-dev, rocm-libs, rocm-cmake, miopen-hip and rocblas) to be installed in the Docker container.

https://github.com/slashml/amd_inference