Deploy an API for Llama 70B in 5 Clicks
Runpod Affiliate Link https://tinyurl.com/yjxbdc9w
One Click Runpod Template https://runpod.io/gsc?template=6e9yxs...
Purchase GitHub Repo Access https://buy.stripe.com/9AQ28UcWh4PF1c...
Detailed API/server setup instructions
Guides for RunPod and EC2/Ubuntu instances.
Forum for questions/support.
Underlying tools:
text-generation-inference: https://github.com/huggingface/text-g...
chat-ui: https://github.com/huggingface/chat-ui
0:00 Deploying an API for Llama 70B in five clicks.
1:27 Making calls to the Llama 70B api
2:24 Calling the API from a chat-ui interface
5:30 Pro Tip! Increasing inference speed
6:07 Running long context models in 5 clicks
6:49 Making multiple api calls for higher throughput
8:35 Why run your own Llama 2 server?
9:23 Github Repo Access