Home Artists Posts Import Register

Content

I got some requests from people who want to dig into my Kuro chat app. Requesting a tutorial on how to use RunPod.


RunPod lets you rent GPU's and use them over the cloud. They charge an hourly rate while you have your Pod running. Generally around 40 to 80 cents an hour. When you are finished with your pod you stop it and don't have to pay anything other than the disk rental fee. Which are a few cents per hour.


Now this is a very deep topic and there's many ways to approach this. Consider this a TL;DR tutorial that covers the basics to help you get up and running. Even though it is quite extensive, it only scratches the surface. If you want to learn more I encourage surfing around the Pygmalion_AI Reddit or Discord channels.



Link to TheBloke's GitHub

Part 1

Part 2

Part 3

Part 4

Part 5



You can also run these locally on your own machine, but that requires a hefty GPU (12GB VRAM or so recommended). However I have zero experience with that so if you're interested you need to look for help elsewhere D:



I hope to have Version 0.2 of my game uploaded in the next day or two. My next Nagatoro animation is also nearing completion!

Files

Comments

NaruNerd100

About the "Exited cost". Will it only drain your deposited balance? Say I put in 10 dollars and it uses all of it, will it charge me further or just end my service until I put more money in?

Anonymous

I followed the process step by step but I can't get the Port 5000 to appear is there something I'm missing?

NaruNerd100

Last question, is there a way to delete the models once their downloaded?

Mantis-X

I am pretty sure its the latter. If you run out of balance it will disable your Pod. Unless you sign up for the auto-charge feature, then it will top off your balance whenever you are below a certain threshold.

Mantis-X

Hmm I am not sure, I will look into that the next time I boot my pod up. You could always trash the pod and spin up a new one if you are out of space. However I do believe that costs an up front charge. (not entirely sure though tbh, but it would make sense)

Mantis-X

Hmm hard to say, there's a lot of areas that could go wrong. Are you sure you're using TheBloke's Docker? Also, sometimes the pod runs out of GPUs to allocate. It will let you enable it but you cant use any of the features. It is possible this is the issue. When this happens you either wait it out, or trash the pod and spin up a new one (which I believe costs the hourly rate up front, so be cautious with trashing/spinning up new pods). If the GPUs are all used up RunPod should give you a warning when you click "play" though.

Anonymous

I am using the docker and I'm not getting an warning message indicating the GPU's are used up.

Mantis-X

Your best bet is probably to join TheBloke's discord and poke around/ask there. YOu can find the invite link here at the top of the readme : https://github.com/TheBlokeAI/dockerLLM/blob/main/README_Runpod_LocalLLMsUIandAPI.md

Mantis-X

Also you could try right-clicking the 7860 Port, copy that URL, and replace the number 7860 with 5000, the URL between the two seem to be the same other than that. It might work!