🐣 Please follow me for new updates https://twitter.com/camenduru
🔥 Please join our discord server https://discord.gg/k5BwmmvJJU

Originally posted to HuggingFace by camenduru

Maybe this is the first ever model trained with TPUs and converted to 🧨 PyTorch 🎊🎉 Trained with google cloud TPUs.

Runtime: 3h 26m 44s
Steps: 18000
Precision: bf16
Learning Rate: 1e-6