hiw3c 日报精选
Hacker News Top 网页快照

Dessa:迄今为止构建的最真实的全双工语音模型

原文标题 · Dessa: the most authentic full-duplex speech model ever built

HN ⭐ 186条评论

来源:Hacker News Top · block.xyz

正文为英文,可一键机器翻译(仅首次需要等待)

You learned to talk by listening. So did

We believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence.

Interacting with current voice AI can just plain suck. It turns your voice into text and throws away an incredible amount of information text alone can’t carry. The most powerful AIs in the world can’t hear how you feel. We think that’s ridiculous, leads to a worse experience, and is potentially dangerous.

We trained Dessa from scratch on nothing but human conversation. Sounds! No text, no pretrained models. It learned to hold a real-time conversation simply by listening. As far as we know, Dessa is the world’s first AI you can have a realistic live conversation with that was trained only from human speech.

Dessa advances the frontier of deep learning models and creates a different category with entirely new and interesting behaviors for researchers to discover. How efficiently do they learn language and facts compared to text? How efficiently can they learn new languages? What sorts of In-context learning can they perform?

Let me listen…

Every clip below is a live recording of a person talking to Dessa. Nobody steers it. Dessa picks what to say, when to say it and how, down to its own voice and personality. Subtitles were added afterward.

Talking about the release

It laughs the moment something’s funny, mid-sentence, the way people do. Other models wait for you to finish, then laugh on cue. Ask it to speak up and it gets louder. Nobody taught it that.

Remembering a movie

The person blanks on a movie title. Dessa finishes the sentence. It also knows a surprising amount, considering every fact it has come from raw sound.

An anniversary surprise

A surprise. Dessa gasps “oh my god” under its breath, then answers with what sounds like teary gratitude.

Expressive conversation

Life advice

Dessa in conversation with GPT-Live-1

In this demo, Dessa is talking with GPT-Live-1, a third-party AI model. Both voices are AI-generated. The developer of GPT-Live-1 is not affiliated with Block and has not reviewed or endorsed Dessa.

More recordings will be shared on X soon.

What is this?!

Audio in, audio out. Dessa is a true speech-to-speech model, and everything it knows came from human speech. It’s full-duplex, so it listens and talks at the same time, like you do. It learned from real conversation, so it knows when to cut in and when to backchannel (“yeah!”, “got it!”, “ooh”) to show it’s with you. Inference runs faster than real time, so there’s no awkward wait. And it runs locally on a MacBook Pro M4 Max or better.

It natively hears and speaks emotion, sarcasm, laughter, a frustrated sigh, a voice cracking with sadness. Run speech through text and all of that is gone, which is why most voice AI feels stilted, slow and, worst of all, annoying.

This is crazy!

LLMs need an insane amount of text. Dessa learned language from hundreds of times fewer words (in fairness, LLMs also have to learn C++ and whatever the hell Angular is).

And it had to learn far more than language. Accents, pauses, breathing, sarcasm, humor, inflection, emotion, when to cut in: every one changes what someone means. Say “I didn’t say she stole my CUDA code” out loud, stressing a different word each time. Same words, different meaning every time.

Speech never repeats. Say a sentence twice, exactly the same way, and the data still comes out different. Text is identical no matter who types it. And a single spoken word takes thousands of times more bytes than the same word in text. People learn to talk so easily that we forget how hard it is. For a machine with no text to lean on, it’s brutal.

But, we figured it out. And we figured out how to train Dessa on only one NVIDIA GB200 NVL72.

Wait but why?

Three reasons: curiosity (aka fun), better understanding between humans and AI, and safety. Babies learn to talk without ever seeing a word written down. We wanted to see if a model could too. It can. It’s just very hard.

Speech is the most natural interface to any intelligence, super or otherwise, and today’s voice AI wastes it. It flattens what we say into text and can’t keep up with human timing.

AI keeps gaining power and independence, and it’s still nearly deaf to the emotion in our voices. Imagine being in serious pain while a voice AI has no idea and just keeps going. That’s the safety problem. Dessa is a step toward fixing it.

Dessa who?

We’re a (very) small AI research lab at Block, founded by former members of Dessa, the Toronto deep learning startup. We’ve built deep learning models since 2012 and generative speech models for over eight years. We chase frontier problems others ignore, and we report directly to Jack Dorsey, Block’s cofounder, Chairman and Block Head.

Calling all safety researchers!

A model that learned to talk from sound alone is new territory, and we want outside eyes on it. We’re giving a small group of AI safety researchers early access to find the risks we haven’t anticipated. If that’s you, .

Team Dessa

More samples

Artist Dessa

Bathroom disaster

Hobbies talk

Lazy AI gaming

Marvel gaming

Noisy GPT live

In this demo, Dessa is talking with GPT-Live-1, a third-party AI model. Both voices are AI-generated. The developer of GPT-Live-1 is not affiliated with Block and has not reviewed or endorsed Dessa.

Oh my god

Poor Elina

Refund

Ronaldo

Safety lab

Singing nonsense Spanish

Spontaneous AI

Wise Dessa

Zelda discussions

Privacy Notice

This Privacy Notice describes how Block, Inc. and our affiliates (collectively, “Block,” “we,” and “us”) collect, use, disclose, transfer, store, retain or otherwise process your information when you interact with the dessa.com website (the “Services”).This Privacy Notice applies only to information collected in connection with your access to and use of our Services. Please read this Privacy Notice carefully. By continuing to interact with our Services, you are consenting to the practices described in this Privacy Notice.

This privacy notice does not cover the processing of your information when using other Block websites, products, or services, such as Square, Cash App, Afterpay, or TIDAL. Each of these offerings is governed by its own applicable privacy notice.

We may also include links to other websites, including social media platforms. For example, you may be given the option to share a news or blog article on a social media platform. This privacy notice does not apply to the data processing practices of other websites, and we encourage you to read the privacy notices available on any other website you visit.

Additionally, this privacy notice does not cover information processed about candidates for employment or contractual services. For details on privacy practices related to employment or contractual candidates, please click here .

INFORMATION WE COLLECT ABOUT YOU

We collect information about you when you interact with the Services or or when you voluntarily choose to provide information to us directly.

Information You Provide

We collect information from you, including:

Identification Information. Your name; email address; mailing address; and phone number.

Other Information You Provide. Information that you voluntarily provide to us, including correspondence with us, or any other actions performed on the Services.

Information We Collect Automatically from Your Use of our Services

We collect information about you and the devices you use to access the Services, such as your computer, mobile phone, or tablet. The information that we collect includes:

Device information. Information about your device, including your hardware model, operating system and version, device name, unique device identifier, mobile network information, and information about the device’s interaction with our Services.

Internet or other electronic network activity information. Information about how you use and interact with our Services, including your access time, browser type and language, country and language setting on your device, Internet Protocol (“IP”) address, the domain name of your internet service provider, other attributes about your browser, mobile device and operating system, features you use, and the date and time of your visit to or use of the Services.

SOURCES OF INFORMATION

We only collect (and/or have collected during at least the 12-month period preceding the effective date of this Privacy Notice) information directly from you, and from how you use our Services.

HOW WE USE YOUR INFORMATION

We may collect, use and share (or have collected, used or shared during at least the 12-month period preceding the effective date of this Privacy Notice), information about you for the following reasons:

Providing, Improving, and Developing our Services

Determining whether the Services are available in your country;

Providing, maintaining, and improving our Services;

Developing new products and services;

Delivering the information and support you request;

Improving, personalizing, and facilitating your use of our Services;

Measuring, tracking, and analyzing trends and usage in connection with your use or the performance of our Services; and

Otherwise providing you with the products and features you choose to use.

Communicating with You About our Services

Sending you surveys and getting your feedback about our Services; and

Sending you information we think you may find useful or which you have requested from us about our products, services and other company updates, such as when you sign up for investor email alerts.

Protecting our Services and Maintaining a Trusted Environment

Conducting investigations, complying with and enforcing any applicable laws, regulations, legal requirements, and industry standards, and responding to lawful requests for information from the government or to valid legal process;

Contacting you to resolve disputes and help you with our Services;

Debugging to identify and fix errors that impair how our Services function;

Enforcing our Terms of Service or other applicable agreements or policies;

Investigating, detecting, preventing, recovering from, or reporting fraud, misrepresentations, security breaches or incidents, or other potentially prohibited, malicious, or illegal activities;

Protecting our, or your rights or property, or the security or integrity of our Services;

Verifying or maintaining the quality and safety of our Services; and

Other Uses

For any other reason we may tell you about from time to time.

WHEN AND WITH WHOM WE SHARE YOUR INFORMATION

We may share the information described in the “Information We Collect About You” section with the following categories of service providers and third parties:

With our Affiliates and Group Companies

With our group companies and corporate affiliates, such as Cash App, Square, Afterpay, or TIDAL, for the reasons outlined above. For example, we may share your information internally to understand how you engage with Block company products to help make our Services better for you and for everyone, and to help us build Services tailored to your preferences.

With Our Service Providers

With service providers who help us to provide, maintain, and improve our Services, including service providers who access information about you to perform services on our behalf (e.g., vendors who help us with fraud prevention, identity verification, and fee collection services), and other entities that help us provide the Services.

Business Transfers and Corporate Changes

To a subsequent owner, co-owner, or operator of one or more of the Services; or

If we do or try to do a corporate merger, consolidation, or restructuring (including during due diligence and negotiation of these); the sale of substantially all of our stock and/or assets; the financing, acquisition, divestiture, or dissolution of all or a portion of our business; or other corporate change.

Safety and Compliance with Law

If we believe that disclosure is reasonably necessary (i) to comply with any applicable law, regulation, legal process or governmental request (e.g., from creditors, tax authorities, law enforcement agencies, in response to a garnishment, levy, or lien notice, etc.); (ii) to establish, exercise or defend our legal rights; (iii) to enforce or comply with our Terms of Service or other applicable agreements or policies; (iv) to protect our or our customers’ rights or property, or the security or integrity of our Services; (v) for an investigation of suspected or actual illegal activity; or (vi) to protect us, users of our Services, or the public from harm, fraud, or potentially prohibited or illegal activities.

With Your Consent

At your direction or as described at the time you agree to share; or

When you authorize a third party application or website to access your information.

With Others

With other companies who deliver services on our behalf, including via the use of cookies and similar automated technologies.

Aggregated and Anonymized Information

We also may share (within our group of companies or affiliates, or with service providers or other third parties) aggregated and anonymized information that does not specifically identify you or any individual user of our Services.

COOKIES AND OTHER SIMILAR AUTOMATED TECHNOLOGIES

When you interact with our online services, or open emails we send you, we obtain certain information using automated technologies, such as cookies, web server logs, web beacons, and other technologies. A “cookie” is a text file that websites send to a visitor’s computer or other internet-connected device to uniquely identify the visitor’s browser or to store information or settings in the browser. A “web beacon,” also known as an internet tag, pixel tag, or clear GIF, is a tiny graphic image that may be used in our websites or emails.We use these automated technologies to collect your device information, internet activity information, and inferences as described above.

Remember your information so you do not have to re-enter it;

Track and understand how you use and interact with our online services and emails;

Tailor our online services to your preferences;

Measure how useful and effective our services and communications are to you; and

Otherwise manage and enhance our products and services.

We set some of these automated technologies ourselves, but others are set by third parties who deliver services on our behalf. For example, we may use other companies’ web analytics services, which use automated technologies to help us evaluate how customers use our websites.Your browser can alert you when cookies are placed on your device, and how you can stop or disable them via your browser settings. Please note, however, that without cookies all of the features of our online services may not work properly. If you use a mobile device, you can manage how your device and browser share certain device data by changing the privacy and security settings on your mobile device. You can learn more about cookies and how to manage your preferences by visiting http://www.allaboutcookies.org.

HOW LONG WE KEEP YOUR INFORMATION

We generally keep your information as long as reasonably necessary to provide you the Services or to comply with applicable law. However, we can keep copies of information about you and any transactions or Services in which you may have participated for a period of time that is consistent with applicable law, applicable statute of limitations, or as we believe is reasonably necessary to comply with applicable law, regulation, legal process, or governmental request, to detect or prevent fraud, to collect fees owed, to resolve disputes, to address problems with our Services, to assist with investigations, to enforce our Terms of Service or other applicable agreements or policies, or to take any other actions consistent with applicable law.

YOUR PERSONAL INFORMATION RIGHTS

Depending on the jurisdiction in which you reside, you may be entitled under applicable law to request:

Access to your personal information, including: (1) the categories of personal information described above that we have collected about you and the categories of sources from which we collected such personal information; (2) the business or commercial purposes for collecting or sharing such personal information; (3) the categories of personal information about you that we have disclosed to third parties for a business purpose; (4) the categories of third parties to whom we have disclosed such personal information; and (5) in portable format, the specific pieces of personal information we have collected about you;

Deletion of the personal information we have collected from you;

Correction of the personal information we have collected about you;

To opt-out of the sharing of your personal information for purposes of targeted advertising.

To the extent you would like to send us a request to exercise your access, deletion, or correction rights, you may do so by contacting us by email at privacy@squareup.com , by writing to us at Block, Inc. (Attn: Privacy), 1955 Broadway, Suite 600, Oakland, CA 94612, U.S.A, or by calling us at 1.844.213.7377.

To help protect your privacy and maintain security, we take steps to verify your identity before granting you access to your personal information or complying with your request.

You may designate an authorized agent to make a request on your behalf as permitted under law, though before we process that request, we will require that you provide the authorized agent written permission to do so and verify your identity directly with us.

You, or an authorized agent you designate to submit a request on your behalf, may also contact us at the address below.

Although some of the information we collect and process about you may be considered sensitive personal information, we only process such information for purposes authorized by law, such as to provide you with certain services you request from us.

We will not discriminate against you if you exercise these privacy rights, nor will we deny, charge different prices for, or provide a different quality of goods or services if you choose to exercise these rights.

Additionally, we do not sell your information to third parties.

Do Not Track

Certain web browsers allow you to instruct your browser to respond to Do Not Track (“DNT”) signals to websites you visit, informing those sites that you do not want your online activities to be tracked. At this time, our websites are not designed to respond to DNT signals or similar mechanisms from browsers.

CHILDREN’S PERSONAL INFORMATION

Our Services are general audience services not directed at children under the age of 13. If we learn that any information we collect has been provided by a child under the age of 13, we will promptly delete that information.

HOW WE