ODSC Webinar | Inference Benchmarking of Prominent Open-Source Large Language Models (LLMs)
In the upcoming webinar, we delve into the inference benchmarking of prominent open-source Large Language Models such as the 13B and 70B Llama-2. We have used a diverse range of compute shapes available inOracle Cloud Infrastructure (OCI), like Intel, AMD, ARM CPUs, and NVIDIA GPUs.
A core aspect of our discussion will center on the crucial metrics of Tokens per Second and the corresponding latency, which are pivotal in evaluating the performance of these LLMs. These metrics not only provide insight into the efficiency and speed of model inference but also serve as key indicators for optimization.
Throughout the webinar, we will guide the audience through a comprehensive journey, covering the various stages of optimization for these LLMs. This includes an in-depth look at the unique challenges and solutions associated with each hardware platform. Our step-by-step process will highlight practical strategies and tweaks that can significantly enhance the performance of these models.
More info: https://app.aiplus.training/courses/I...
→ To watch more videos like this, visit https://aiplus.training ←
Do You Like This Video? Share Your Thoughts in Comments Below
Also, You can visit our website and choose the nearest ODSC Event to attend and experience all our Trainings and Workshops:
https://odsc.com/boston/
Sign up for the newsletter to stay up to date with the latest trends in data science: https://opendatascience.com/newsletter/
Follow Us Online!
• Facebook: / opendatasci
• Blog: https://opendatascience.com/
• LinkedIn: / open-data-science
• Twitter: / _odsc