If you're participating in the Deepgram Hackathon on DEV, weβre so excited to have you joining us! Need some help with your submission or participa...
For further actions, you may consider blocking this person and/or reporting abuse
@michaeljolley
I'm trying to send real-time audio to deebgram and get the transcription but I get this error while doing so.
I'm using pyaudio to send the binary data.
input_audio = stream_in.read(3200)
await ws.send(input_audio)
DEBUG:websockets.client:< CLOSE 1008 (policy violation) DATA-0000 [11 bytes]
Hi @sandy_codes_py, I'm not a Python pro, but I can hook you up with @tonyasims. She's an amazing pythonista and one of Deepgram's Developer Advocates.
One thing that might help, is there a GitHub repo where we can review the full block of code?
Please check here
I've used api reference code from the deepgram docs
Sweet. Thanks for that. Tonya is out of the office today, but she'll likely respond Monday.
Mental note: I should really start playing with Python more so I can help more. π
That would do!
I did find a workaround it by using her post "Live Transcription With Python and Flask" but that's not really needed here.
I just want to run it locally to do something cool.
Yay, even I've to learn about async and how it works.
Great! I think she wrote several posts like that: "using Flask," "using Django," "using FastAPI," and more.
And yeah! for learning! ππ
Hi @sandy_codes_py ! Happy Monday! I'm sorry to hear you were having some trouble. Were you able to figure out the issue? Please let me know if you still need some help and we can work through it together.
Nope, I followed your Flask repo and did that same. But I really want to run that in just Python and not on Flask. I do have limited knowledge on asyncio. Will be learning that soon. I've shared the error and the complete code I'm using the same thread.
Ok, so if I understand correctly you follwed the Flask example in the tutorial but still having some issues? Is the issue still with PyAudio?
Also, is this the error message you received? (I want to make sure this is the correct error):
input_audio = stream_in.read(3200)
await ws.send(input_audio)
DEBUG:websockets.client:< CLOSE 1008 (policy violation) DATA-0000 [11 bytes]
Sorry to misled you!
Actually I wanted to directly send the audio feed using PyAudio to the Deepgram websocket (that's when the above error is occurring).
But I found your Flask tutorial and used that instead which worked in the first go.
The complete code I used can be found here.
Oh nice! Good to hear you found a solution with the tutorial π.
About asyncio, yea, I totally understand it can be very confusing. It took me awhile to wrap my head around it. How do you plan on learning asyncio? Tutorials? Blog posts? Videos? Something else?
Gonna read through this for a while and try some hands-on stuff.
docs.python.org/3/library/asyncio....
I'll try to make a tutorial here once I get a good grasp.
Wonderful! Make sure to let me know when it's published so I can read it :)
You got it!
I want to participate, but, the post says that it's
Open only to 18+.
So, does that mean I'm not eligible ? :(
I don't think it means that you aren't eligible for badges for helping. So if you contribute in the Help and Community threads, I think that means you can still earn badges.
i have a couple ideas, if i just write a post with he ideas, would I be eligible? (I could even execute them)
The post seems to imply that all parts of the Hackathon are 18+ to participate. It is likely there for legal reasons, and I would think that if you should be able to participate if your parents approve.
Hello! Unfortunately, DEV hackathons are currently only open to community members 18 years of age and older. We would absolutely love to open this up to younger community members, but we are legally unable to at this time. We revisit the topic frequently as a team and intend to open up our challenges to younger folks if and when it's possible for us to do so. I'm so sorry to bear this news. We really value having all of you as part of DEV.
It's normal for contests to be 18+ due to prizes and legal ruling.
You can ask the dev people if it's OK if you submit but are not considered as a potential winner.
Yeah, I've already submitted my project. It's completely fine if I don't win, or win and not get prizes. atleast I got to learn stuff, that's all that matters
That's a great attitude @dhravya! I'm REALLY looking forward to checking out your submission.
Hi there! Thanks a lot!
Here's the post dev.to/dhravya/deepsubtitles-gener...
Well that was fast, great job ππ
thanks a lot!
Hi,
I would like to participate with a .NET solution using your new .NET SDK.
This should use the audio from the microphone to implement a voice dialog within the app (kind of a bot and some speech control for the app).
Unfortunately, I could not find any sample how to connect the Microphone stream with the CreateLiveTranscriptionClient.
Can you give me some hint show to connect the mic audio with the SendData function?
Thanks Frank
Hi Frank, do you have a stream of audio to send? As in, have you already captured audio from the mic?
Hi Michael,
I can get the audio in from the mic and write it to a file with the help of the NAudio package..
There you can see the code that writes the input to a file. It is in the event handler OnDataAvailable but as a comment.
But this is the same method that has the call SendData to send data to the deepgram service.
I have posted my code in this repository github.com/FrankPohl/HealthAssista...
This is a MAUI app because besides console apps, this is the only platform with .NET 6 support.
I use this because my plan was to build an app to help handicapped people to put in data like their blood pressure with speech. Therefore I need a UI and a permanent connection from the microphone to the recognition service.
Btw., you should have mentioned in your sample code with the logging what nuget packages you have used. Took me some time to figure out why "AddSerilog" did not work.
I can add more information about the format from the Microphone stream.
Samplerate: 48000
Encoding: IeeeFloat
Bits: 32
Channels: 2
Blockalign: 8
Bytes per Second: 384000
Must this be converted or should it be processable from the interface?
I added the file test.wav to the repository. This file was recorded from my microphone.
I also added a function to send a file for conversion to the deepgram service.
Trying to convert this raise error 400 (Bad Request).
I hope someone can come up with a solution so that I can continue to work on my contribution to the hackathon.
Thanks @fp. I've been playing with it yesterday and realized it was IeeeFloat. The API doesn't support that format. You'll need to convert it to something else. I noticed in your request you're telling the API you're sending Linear16. That's fine once the raw audio is converted to that.
Hello Michael,
I made a conversion for my input.
I do not get an error when I use the CreateLiveTranscriptionClient but I do not get any result from deepgram but I cannot find out what the problem. Maybe you can have a look? I updated the github project witht he latest code samples.
If I try to convert a file that is PCM with a sample rate of 16000 I get an error from the SDK. The file is TestConverted-16000-Pcm-2.wav and is a recorded and converted audio sample from me.
Would be nice if you could help me with that again.
Frank
I'm looking at this tonight and will let you know something soon!
Okay. I created a quick .NET 6 Console app and have it working based on a slightly modified version of your code. You can find it at gist.github.com/MichaelJolley/b52f...
Hope that helps!
Thanks for your help Michael.
What a shame to bother you with such a stupid error in my code.
I would like to publish that program as an example on Git, or will you do that?
No way @fp! It took me a while to figure it out so don't feel bad at all. I'm glad it's working for you now.
You can certainly publish it if you'd like.
Hello @michaeljolley and DG Team π
I just wanted to know do we have any examples where we are writing the transcripts to a separate output file instead of showing the complete json on the console. And by transcript, I mean the only part in the json output that contains the text sentences.
Moreover, let me know how to make sure we scan through the complete length of the audio we are passing instead of just a part of it.
Thanks in advance.
Hiya @21rutam! Good question. Let me make sure I give you the right answer.
First, are you using one of our SDKs? If so, which one?
Second, are you trying to save the whole payload or just the words?
Yeah, I am using NodeJS to do the job. I want to just save the transcript part of the response payload. And do let me know if we can define the duration of the audio for scanning (Complete audio or Just for a timeframe)
CC: @michaeljolley
You can't define the duration. The API will try to transcribe the entire audio file every time, not just a section.
Using the Node SDK, you could send your request with the
utterances
feature turned on. (e.g.utterances:true
)Then, when the transcription comes back you can use the
.toSRT()
or.toWebVTT()
functions to generate a text based version of the transcript with timestamps. Then you'd want to save it locally usingfs
.Example:
That was real quick. But I don't really want it like a subtitles file. Rather I just want the transcript text to be saved to the file and not anything else. I am talking about the sentences in that
transcript:
part (the one marked in purple).CC: @michaeljolley
You could just use the transcript property itself:
Thanks a bunch @michaeljolley . You rock ππ
Hello guys,
I'm using the Node.js SDK to stream an audio file from a web application. I'm using Typescript and was wondering why I receive the following
ts
error here.Do I really need the
version
property here? Could anything unexpected happen if I don't?My current code looks something like this (and works file if I set
@ts-ignore
on top of it):Great find. That's a bug in our SDK.
version
is an optional parameter.New release is on NPM now with that fix. Thanks for catching & reporting it.
That was quick. Thanks :-)
Having a little bit of a hard time figuring out the final part of the
listen
endpoint.1) In a local environment, how would I go about sending the empty binary message to the server?
2) Upon ending, where is the final transcript JSON now?
Thank you.
Great question. It depends on how you're hitting the API. If you are using the Node SDK you'd use the
finish
function, as in:If you're communicating with the WebSocket directly with JavaScript you can send a new Uint8Array, as in:
When the Deepgram API receives that it will finish transcribing the audio, send a final transcript, and then close the WebSocket connection.
Appreciate the response.
What about good old Python please?
P.S. Great Twitch btw, really enjoyed it.
Thanks!
For the Python SDK:
Against the WebSocket without the SDK would be something like:
Thanks for coming to the stream today!
@michaeljolley I create a sample to send audio directly to deepgram from the microphone.
I have created a repository for this sample here github.com/FrankPohl/DeepGram.NETS...
But there is one thing I do not understand. I resample the audio input to PCM with a sample rate of 16000. But in the deepgram options is 44100 given as a sample rate. If I change that to 16000 I do not get a transcription. Why is that?
That's a REALLY good question @fp! That code isn't resampling, it's converting it from 32-bit to 16-bit. IeeeFloat is a 32-bit format. We're basically converting C# long into C# short. This blog post does a good job at describing the differences in the two.
@michaeljolley Which code is not resampling? The code in my example on Github converts from to short, that's right. But in a second step it does resampling because I'm averaging 3 consecutive input values into one output value. I thing this means that the sampling rate is reduced from 48000 to 16000. The wav file that is written in parallel has this sampling rate and sounds alright.
Hi! First of all, Love the idea of this hackathon! Thanks for hosting, Deepgram :)
I have question about real-time transcript with deepgram(node.js SDK).
I tried this tutorial below
developers.deepgram.com/documentat...
I am wondering if there is a way to use 'opus stream' audio instead of 'url' here in the tutorial for transcription??
Thank you!
Solid question. Normally you wouldn't want to send a stream from a URL in, you'd be accessing a microphone and streaming that audio in. I haven't tried opus stream specifically, but you can certainly try it. Is there a reason you need opus stream? That seems like an unusual format for live streaming.
Thanks for reply Michael! yeah We use opus because we are building a real time transcription discord bot for deaf gamers! And discord.js uses opus as a formatβ¦ My teammates gave up in the middle so I donβt think I can finish it by 11th but still wants to get this one done :)
That's amazing! So I think you may can stream that in but you'd want to make sure you specify the encoding, sample rate, etc. in your request.
I was treid to make a Live speech Transcript project, but failed due to unknown error. Can someone please help me on that?
This show in the browser console:
This is error
Here is my GitHub link: github.com/mhasanmeet/DEEPGRAM-liv...
Nice! I can help with that.
It looks like the credentials you're sending are incorrect.
That code looks like you're sending the API Key Id rather than the API Key. When you create an API key, be sure you're copying the key itself. You should see a screen like below. You can click that copy icon to copy the actual key. Be sure to copy it, because you won't be able to see it again for security purposes.
Thanks Micheal, I'm gonna try it again. I'm really exicited about this Deepgram project. I have so much idea, I will implement one by one. And thanks, you guys already made some awesome demo projects, it really helpful!
Awesome! I can't wait to see what you build. Be sure to reach out if I can help!
I created Deepgram account. But How do i
Select one of the following four categories
I don't see any of this options on the Deepgram website
The instructions said "Select one of following categories," but it isn't clear that you're really selecting it in your mind. You're deciding what category you want to build a project for. When you create a submission you'll use this template to create your submission post. That template has a section where you'll enter what category you're submitting your project for.
Thanks, It's helpful
Hi,
I'm using the DeepGram standard recognition service without an additional model.
I have a problem with the recognition of "weight" and "wait". Often "wait" is recognized instead of "weight". Is there a way to enhance the probability for "Weight" because in the use case it is true for 99 percent of the cases?
Or is this just a problem of me not being a native speaker?
Frank
That's an amazing question @fp! You can add some "weight" to the word "weight". (See what I did there. π)
You'd want to use the
keywords
feature. If I remember correctly, you're using the .NET SDK, so here's how you'd set that up:Thanks @michaeljolley that's exactly what I was looking. Unfortunately this raises :
Deepgram Error: Unexpected character encountered while parsing value: [. Path 'keywords', line 1, position 82.
when the StartConnectionAsync method is called.
The error is raised in line 44 in Helpers.cs
t = JsonConvert.DeserializeObject>(json);
How do you guys at Deepgram deal with the data provided by users? Is there any document describing it?
There is information about the data collection in the Terms of Service Agreement, the Privacy Notice, and Data Privacy Policy.
From what I can tell, uploaded data is collected and stored, measures are in place to protect the data, and customers will receive an email or message about any security notices.
Note: I am not a member of the Deepgram team, so any clarifications should be directed towards them.
Got it, thanks for the answer.
Please read through the Terms of Services and the Privacy Policy on questions related to data retention. In short, Deepgram stores and processes a small percentage of the data for the purposes of improving services and/debugging debugging client requests. If you have further inquiries, including one-off requests for data deletion, please reach out to the Deepgram customer success team at support@deepgram.com.
In the case of a breach, end users will be notified within 72 hours of breach confirmation.
What do you mean by 'deal'? Just making sure we can answer your question correctly :)
I want to understand if you guys store this kind of information that is given by your users (input audio). If it's stored, how you manage to keep it safe from public access. And if, for some reason, this data leak, what you guys will do, for example, alert people contacting through e-mail.
I'd be curious to know where to look for any data retention policies concerning those of accidental streaming transcriptions when someone forgot to turn off their mic.
Hi all, just wanted to ask about is there any API/Feature in DeepGram to convert any realtime audio language to any text language. Basically like, if i want to convert e.g chines realtime language to english audio.
Hey! We have language support, but to do the translation, you'd need to use a translation api like itranslate. We have a video and repository up with an example of how to do that: Live Transcription Badge Video
Hello there! I hope everyone is doing well and having fun. I am having it for sure!
But I need some help... If I think I've posted a submission under the wrong category. Is it OK if I edit the submission post and change the category? The first time I read the categories description I didn't fully got the idea π
Thanks in advance!
Absolutely! You can make changes until midnight UTC on April 11th. So feel free to tweak until you're satisfied!
Can we have multiple submission for this hackathon?
The post says yes and even encourages multiple submissions, so yes i guess
Yes! We'd love to see you get as many chances as possible to win!
I wish I'd discovered this thread sooner. I had a lot of questions and bugs.
There's still a few days. Anything I can help with?
Yes please...how to hide the api key so that it cannot be exposed to the public.
Sure thing. @_phzn wrote a great blog post on doing just that.
developers.deepgram.com/blog/2022/...
Hey there @michaeljolley, hope you all are doing great!
Just curious about the Article Template DEV has presented. I think I can change the heading and the other things according to need, and of course, including the everything mentioned.
Am I right? Thanks!
You're 100% right!
I have one interesting question here,
i just started exploring deepgram, i wonder how deepgram is different than other STT Python libraries out there ? as there are so many opensource python libraries are available for example github.com/mozilla/DeepSpeech !
Hi @leo1612d! Thank you for your question. I'm not as familiar with DeepSpeech, the mozilla library you posted so I can only speak to Deepgram.
At Deepgram our mission is we believe every voice should be heard and understood.
A major advantage Deepgram has over other STT libraries is that we're the only provider that uses End-to-End Deep Learning. This means that you'll get improved accuracy with training, an easier to use and integrated API and is priced to use at scale.
Hi all I want to ask a question , I am using this API to convert the live user audio to text
"api.deepgram.com/v1/listen?languag..." here is a param for language, which i am assuming that if i provide "zn-ch" value it will get the audio as china and convert it to text, but its not working & i am assuming that this happing because of i select English when i was creating token for my deepgram api, is their any way to create dynamic token on runtime when user choose the language from FE.
Hey Zain! You can use the same token for multiple languages. Is there code you can share and maybe an error message? We can also look at it live on Twitch tomorrow at 1:30pm EDT if you're up for it.
here is the git repo github.com/zaahmed-nisum-com/dev.t.... Thanks for the reply and yeah i just check it was my mistake the language is now working for multi language. thanks again
Hi all,
Can we use deepgram in react.js ?
You can use Deepgram with React. Our primary tech stack uses Vue, so many of our blog post examples do as well. You can take a look at those and see that React would work as well.
Thanks, got it.ππ₯
Hello. Are we allowed to create a API wrappers using Deepgram, in this hackathon ?
You just need to use Deepgram in your code for this hackathon mostly.
By API wrapper, are you talking about an SDK? I'd love to hear more about this idea.
How can I use multiple keywords feature with Deepgram SDK?
Hey @gulshanaggarwal! Have you checked out the keywords docs section on Multiple keywords?
You can add them as an array.
So it would look something like this:
Oh! yeah, thanks for sharing.
Hey @bekahhw actually the link you mentioned(for keywords feature) only provides implementation through API call, not for SDK however I could find it on my own.
Anyone here has implemented a real-time speech recognition with Python with Deepgram Python SDK or with API???
Hi Santhosh! Are you looking for project ideas? What did you have in mind?
Just wanted to check if someone has successfully connected API version of the websocket.
I'm already done with my project.
This is a really cool hackathon. Thanks for putting it on everyone. Good luck to all the participants.
Thanks, Michelle! We're pretty excited for it.
How easy or straightforward is it to integrate with a text based chatbot? Any examples?
Good question. I suppose it depends on what you're trying to do. Because Deepgram is a speech-to-text API, are you looking for users to ask a question to the bot rather than type? If so, that's pretty straightforward. You can get the transcript of the person speaking and send that text to the chatbot.
I don't know of any specific examples of sending it to a chatbot, but we have several examples of taking real-time audio and transcribing it. That link will show several pieces of documentation and many blog posts that are using it.
Yes mostly on the usecase - talk to bot instead of type; any integrations with intent classifiers (RASA/Diaglogflow)?
I don't think we've built a demo for those yet. Very interesting. I'd love to see that in action.
Hi, can we submit one project under multiple categories?
I guess so. But it should be in one category mostly tho.
Okay, thanks!
IIRC from previous hackathons, you're allowed to have multiple submissions. Is that still true? And if so, does that apply to a mix of code and "Innovative Ideas" submissions?
Absolutely! We're stoked to see all the cool submissions. And we actually talked about finding ways to optimize your submissions for multiple categories on our Twitch kickoff stream today|!