>Llama Workspace connects to AI providers like OpenAI and Anthropic via API. In the API, each word for the most commonly used model (GPT-4o at the time of writing) costs approximately $0.000025. Therefore, to break even with the ChatGPT Teams license, which costs $30/seat/month, you would need to generate 1,200,000 words.
I would debate this one. As a "normal" user of chatgpt with around 10-20 questions a day, I can exceed this $30 per month because
1) it's not a word per se but token. A lot of the words are tokenized.
2) for each chat, for each question all the context above is re-sent and counts for cost.
I think pivoting towards self hosting llama models and charging for them might be more profitable to you and to the users.
The data we provide about savings is based on the actual usage of our early adopting companies. It includes data from marketers, legal counsels, developers and support agents.
So for example, your scenario of 20 questions a day for 30 days:
- Monthly queries 20*30 = 600 queries/mo
- Model gpt-4o ($2.5/M input, $10/M output)
- Average input tokens 1,000 (actuals are much lower)
- Average output tokens 400
=> Cost per run $0.0065
=> Monthly cost $3.90
PS: True, We deliberately used "word" and not token, in some context, so that anyone can easily understand it.
(My company selected LibreChat to host our internal “GPT Teams” implementation after surveying and evaluating the current state of tools as of about six months ago, but I’m sure it’s a rapidly evolving space.)
Thanks for the question! I was kind of expecting it :)
It is indeed a rapidly evolving space and I think choosing LibreChat was a great choice. We only have good words for them!
In our case, we'd like to put our efforts on addressing specific needs of businesses & organizations that aren't always well covered by similar open source projects. Foundational things like granular permissions, advanced auth, usage reports, spend control and content moderation are top priorities in our roadmap. And then, we'd like to invest heavily in data integrations and RAG.
Finally, we'd like to make it extremely easy to self-host by making the deployment and upgrading process a breeze.
Hope this takes us to a place where people find value in our work!
Depends on usage, but the stack which is essentially Nodejs + Postgres is relatively low footprint. The most demanding workload is "next build", which you can do outside your server.
As a rule of thumb, you could start with the following setup:
if you run "next build" on the server: 2CPU / 4Gb RAM
if you upload the built artifacts to the server: 1CPU / 2Gb RAM
I'd say that the main difference is that openwebui is not so focused on making the product "teams first". Yes, it has users, but access to data like documents uploaded for RAG are shared across all the users in the same workspace.
We are going in a direction where access to data is scoped with permissions. We do it via apps (same as "GPTs" in ChatGPT). As a user you upload documents to an app, and then have an advanced sharing system to control who has can access it.
We will be doubling down on making company documents and data accessible from the app, which we aren't sure it's a priority got Openwebui.
>Llama Workspace connects to AI providers like OpenAI and Anthropic via API. In the API, each word for the most commonly used model (GPT-4o at the time of writing) costs approximately $0.000025. Therefore, to break even with the ChatGPT Teams license, which costs $30/seat/month, you would need to generate 1,200,000 words.
I would debate this one. As a "normal" user of chatgpt with around 10-20 questions a day, I can exceed this $30 per month because
1) it's not a word per se but token. A lot of the words are tokenized.
2) for each chat, for each question all the context above is re-sent and counts for cost.
I think pivoting towards self hosting llama models and charging for them might be more profitable to you and to the users.
The data we provide about savings is based on the actual usage of our early adopting companies. It includes data from marketers, legal counsels, developers and support agents.
So for example, your scenario of 20 questions a day for 30 days: - Monthly queries 20*30 = 600 queries/mo - Model gpt-4o ($2.5/M input, $10/M output) - Average input tokens 1,000 (actuals are much lower) - Average output tokens 400
=> Cost per run $0.0065 => Monthly cost $3.90
PS: True, We deliberately used "word" and not token, in some context, so that anyone can easily understand it.
Thank you and congratulations!
How (in your opinion) does this project compare to https://www.librechat.ai?
(My company selected LibreChat to host our internal “GPT Teams” implementation after surveying and evaluating the current state of tools as of about six months ago, but I’m sure it’s a rapidly evolving space.)
Thanks for the question! I was kind of expecting it :)
It is indeed a rapidly evolving space and I think choosing LibreChat was a great choice. We only have good words for them!
In our case, we'd like to put our efforts on addressing specific needs of businesses & organizations that aren't always well covered by similar open source projects. Foundational things like granular permissions, advanced auth, usage reports, spend control and content moderation are top priorities in our roadmap. And then, we'd like to invest heavily in data integrations and RAG.
Finally, we'd like to make it extremely easy to self-host by making the deployment and upgrading process a breeze.
Hope this takes us to a place where people find value in our work!
Very cool!
What is the recommended resource usage for self-hosted? I don’t see it in that section on your site.
Depends on usage, but the stack which is essentially Nodejs + Postgres is relatively low footprint. The most demanding workload is "next build", which you can do outside your server.
As a rule of thumb, you could start with the following setup:
if you run "next build" on the server: 2CPU / 4Gb RAM if you upload the built artifacts to the server: 1CPU / 2Gb RAM
What does this do that ollama webUI cant do? Just curious
I'd say that the main difference is that openwebui is not so focused on making the product "teams first". Yes, it has users, but access to data like documents uploaded for RAG are shared across all the users in the same workspace.
We are going in a direction where access to data is scoped with permissions. We do it via apps (same as "GPTs" in ChatGPT). As a user you upload documents to an app, and then have an advanced sharing system to control who has can access it.
We will be doubling down on making company documents and data accessible from the app, which we aren't sure it's a priority got Openwebui.
hope that answers your question!