Hi. I'm Dima, writing from a city where ChatGPT simply doesn't work. After enough suffering with workarounds and Telegram bots, my team and I decided to build our own interface for it — one that works anywhere.
Despair
ChatGPT by OpenAI needs no introduction, and neither does the fact that in a lot of places it barely works at all. If you haven't tried yet, you'll need to find a VPN it's happy with, rent a virtual phone number it's happy with, and pay $20 from a card it's happy with. You get the idea.
How to use the internet in 2023
Personally, I'm a big fan of intimate relationships with computers, but even I felt sick when keeping a suitable VPN connection up all the time became a necessity. Turns out my server in Poland doesn't satisfy OpenAI or the local streaming service. Well, I could have switched it or paid for something else, but by then there was no stopping me.
The Search
Almost every web app has a server side and a client side. And if you can't use the client properly, why not use the server? Luckily, OpenAI has a clear API for exactly this. And if a common problem has an obvious solution, someone's already gone for it — the first thing that comes to mind is a Telegram bot. There's already a ready-made chat client there; all you have to do is write a simple service that routes requests to OpenAI. Then it's just a matter of topping up the balance now and then so the per-request charges don't drain it to zero.
Turns out there were tons of these. You can buy requests or tokens in them. Some get given away free on bot activation. A token is a piece of a word, a letter, or a whole word that the bot processes. As a rule, one word in a generated response costs a little more than 1 token. It costs more in some languages because of encoding.
Hope
Mostly I need GPT to talk through configurations of different programs, solve problems in several programming languages, write new texts and edit existing ones. A lot of people think ChatGPT is like Google: you ask it something, and it finds matching material, just cooler, straight to the result, carefully thought through.
In reality, you need to train the model so that it can eventually give you the result you actually want. Sometimes you need to write 10 prompts in one context just so the neural network "gets" what's being discussed at all, and only then ask one simple question and get a brilliant answer.
ミーシャ、全部クソ、もう一度やり直そう!
Telegram bots have a /forget command for switching context, which is worth using so the neural network "forgets" what you were talking about earlier and doesn't try to come up with a developmental summer reading program for kids while still "keeping in mind" the specifics of my kubernetes cluster.
Overall things were going fine, until they weren't. During my time with the bots I realized three things:
- I need to be able to quickly return to previous contexts and continue the conversation
- I need GPT-4 because it's insanely smart, and it'd be nice to have it in the same subscription
- I need to be able to watch the neural network type out its answer and be able to somehow stop its stream of "thoughts" on a question I accidentally sent unfinished
And Telegram, unlike the original ChatGPT interface, just can't do that.
Growth
Everything came full circle, and I realized what I actually needed was ChatGPT as it was meant to be — a standalone service.
The best you can find on that front are foreign services like Merlin, which don't accept certain local cards, a bunch of projects on Github you can deploy yourself with your own API key, and a couple of half-finished interfaces someone's already launched but that you can't really use properly.
In situations like this, the first thing that comes to mind is: if I'm not the only one with this problem, I should just build my own solution. I started putting the word out in group chats, messaging friends and friends of friends. That's where I was surprised to find people who were pooling money to buy API access and bolting it onto their 5-10 person companies. In the end a dozen people gathered who told me to stop talking and start charging.
At that point my colleagues at Script Heads, with whom we'd built more than one great product across all kinds of fields, saw the same pain point, and we started building. By then we already had groundwork in chats, payment processing, subscriptions and everything else, plus a solid cluster on Yandex Cloud. On top of that, we're represented in the EU, which means we can settle payments with OpenAI systematically.
The stars aligned, and GPTunneL was born.
It doesn't reach for the stars, doesn't claim to be original, it just does its job. Exactly what's needed. While we were shipping the first stability updates, the idea came up to add other neural networks via API too. That's how Midjourney showed up on the platform. We could stop here, but if you're reading this on a business-focused platform, you're probably curious how it all actually works.
The Numbers
We have a legal entity and a payment processing agreement with a bank. The bank collects money from users, and after that the process looks like this:
Nothing unusual, but doing it solo is a real pain. Next comes spending down the API limits, where you get a monthly credit of a certain volume. You can't go beyond it — you have to negotiate a limit increase with support and wait for a reply. OpenAI charges tokens for:
- The request itself
- Every word in the response
- Every previous message in the conversation history
The price is different for GPT-3.5 and GPT-4. We had to estimate and calculate how much a request should cost, because explaining the billing is genuinely hard. Through trial and error we landed on a scheme where users buy virtual currency — tokens. A request to 3.5, as a baseline, costs 1 token, to version 4 — 3 tokens, and to Midjourney — 15. Tokens are topped up pay-as-you-go in three tiers:
- 25 tokens for $1
- 100 for $3
- 400 for $8
That's how covering all our costs started to add up. We'll see how it goes from here.
A Business?
We're hoping this simple service can build an audience of people for whom using neural networks at work and in life actually matters. Just chatting with the bot isn't the product we'd want to end up with — it's more of a here-and-now fix. Right now we're gathering data by analyzing user behavior to find problems that dedicated neural-network interfaces could solve more productively than a plain chat. If you have any such problems — write them in the comments, we love shipping and continuously improving useful things.