All speakers are unseen during training. Data is from librilight. All models are trained on maximally 30 seconds speech.
Model Architecture of VoiceStar
VoiceStar is an autoregressive voice cloning (zero-shot) text-to-speech model, that can precicely control the output duration and generation speech longer than maximal training duration during inference
To clone an unseen voice or edit a recording, VoiceStar needs only a few seconds of the voice. Trained on 30 seconds speech, VoiceStar can generate 40 seconds long speech.
All speakers are unseen during training. Data is from librilight. All models are trained on maximally 30 seconds speech.
| Reference Speech | Target Transcript | VoiceStar (Ours) | F5-TTS | MaskGCT | Ground Truth |
|---|---|---|---|---|---|
| The tale is but conjecture, yet all the romance is there; for picture a wall over fifteen feet high built as they built long ago, standing for all those ages between two gardens. For would not the temptation arise to peer over the wall if a young man heard, perhaps songs, one evening the other side? | |||||
| That is sure. You do not know anything about the greatest men and women. I went out to write the life of General Garfield, and a neighbor, knowing I was in a hurry, and as there was a great crowd around the front door, took me around to General Garfield's back door and shouted, “Jim! Jim!” | |||||
| I secured two corner seats in a smoking-carriage, and then paced up and down the platform waiting for him. When men have nothing else to occupy their minds, they take to thinking. Having nothing better to do until B. arrived, I fell to musing. What a wonderful piece of Socialism modern civilisation has become! | |||||
| No telegraph ever carried the news faster, all over the region, that Morgan had a wonderful harp. All the grass in front of the house, was soon worn away by the crowds, that came to hear and dance. As soon as Taffy touched the harp strings, the feet of everyone, young and old, began shuffling, | |||||
| "No," said Betty. "We're a walking club." "No politics?" "None whatever." "All right. Now, then, I'll see why Martha didn't come over. I can't understand." "Perhaps this is she now," said Betty, as another woman was seen coming up the walk. "It is," said Mrs. Robertson. "That's Martha Black." The two met. There was much talk, of which the girls caught some, | |||||
| The Princess Sarah cannot chide us now. We're free! I love the wilderness! I love The earth and sky! Look at those birds, Far as the fleecy clouds! And here Are flowers with which to wreathe my bow. With it I'll bring thee deer and fowl to dress, | |||||
| Instead of troubling to answer him Peter flew round the room, taking the mantelpiece on the way. 'How topping!' said John and Michael. 'How sweet!' cried Wendy. 'Yes, I'm sweet, oh, I am sweet!' said Peter, forgetting his manners again. It looked delightfully easy, and they tried it first from the floor and then from the beds, |
All speakers are unseen during training. Data is from librilight. All models are trained on maximally 30 seconds speech.
| Reference Speech | Target Transcript | VoiceStar (Ours) | F5-TTS | MaskGCT | Ground Truth |
|---|---|---|---|---|---|
| The lady of the portrait must have been some twenty odd years; she was no simple maiden, no half opened rosebud, but a woman in the full resplendency of her beauty. Her face was oval, but not too long, her lips full, half open and smiling, her eyes cast a languishing side glance, and she had a dimple on her chin as if formed by the tip of Cupid's playful finger. | |||||
| What! This unknown continent, this virgin soil----" "Has already a name," replied Altamont, coolly. Hatteras was silent, but his lip quivered. "And what name has it, then?" asked the Doctor, rather astonished at Altamont's affirmation. "My dear Clawbonny," replied the American, "it is the custom, not to say the right, of every navigator to christen the soil on which he is the first to set foot. It appears to me, therefore, that it is my privilege and duty on this occasion to exercise my prerogative, and--" "But, | |||||
| "This walking is harder on them than you'd think." Fortunately the garments came on time, and in fresh outfits the girls prepared to bid farewell to the camp, and once more proceed on their way. The boys begged for permission to accompany them, but Betty was firm in refusing. "We said we would make this tour all by ourselves," she declared, "and we are going to do it. Some other time you boys may come along. But there is only another day or so, and we will be back home. Please don't tease." The boys did, but that was all the good it availed them. | |||||
| I knew that it was safe where you had put it, Miss Nelson," and he looked at Betty. "Besides, I have been without it so long now that it seems almost as if I never had it. And from all the good it is going to do me, perhaps I might be better off without it now." "We didn't exactly understand what you meant by the note you wrote," said Betty. "Well, I'll tell you how that was," he said, frankly. "You see, I was left considerable money by a rich relative, but I had bad luck. Maybe I didn't have a good business head, either. Anyhow, | |||||
| This is only a line to tell you all how much father loves you. The Pawnee Indian drew you the picture of the little dog, which runs everywhere round the ship, and now and then howls a little when the band plays. Near Santiago, May 20, 1898. DARLING ETHEL: I loved your little letter. Here there are lots of funny little lizards that run about in the dusty roads very fast, and then stand still with their heads up. Beautiful red cardinal birds and tanagers flit about in the woods, | |||||
| I watched their purple shadows trail Across the sea and hide Within the hollows of the waves That rode the rising tide. Sometimes the little flakes of foam Dashed up in twinkling spray; And out along their silver paths The ships sailed far away. As through the sun I followed them With straining, eager eyes, From out the sparkling waves I saw A shining vision rise. It seemed a ghostly castle white, With battlement and tower, | |||||
| and have obstacle races down the hall when you brought in your friends? Mother continues much attached to Scamp, who is certainly a cunning little dog. He is very affectionate, but so exceedingly busy when we are out on the grounds, that we only catch glimpses of him zigzagging at full speed from one end of the place to the other. The kitchen cat and he have strained relations but have not yet come to open hostility. |
All speakers are unseen during training. Data is from librilight. All models are trained on maximally 30 seconds speech.
| Reference Speech | Target Transcript | VoiceStar (Ours) | F5-TTS | MaskGCT | Ground Truth |
|---|---|---|---|---|---|
| To do this properly it was necessary to engage a carpenter, who would make the necessary framework. Then Thomas and James would help him to put up the building. But money was needed to carry out the undertaking, and this was the purpose to which Thomas decided to devote his first earnings when he left home. One day Thomas returned from an expedition in search of work, in high glee. He had obtained employment in the State of Michigan. He had engaged to assist in clearing the forest, that is, in cutting down trees for a man who was about to make a farm. | |||||
| He would go see him at once. The trouble with Peter is that he doesn't think of all sides of a question. He is impulsive. That is, he goes right ahead and does the thing that comes into his head first, and sometimes this isn't the wisest or best thing to do. So now he scampered down into the Green Forest as fast as his long legs would carry him, to hunt for Prickly Porky. It was no trouble at all to find him, for he had only to follow the line of trees that had been stripped of their bark. "Good afternoon, Prickly Porky. Have you heard the news about Chatterer?" said Peter, talking very fast, for he was quite out of breath. "Yes," replied Prickly Porky. | |||||
| Whitburn stood aside. Handley, Smith and Pottgeiter went through the door; the others followed. The other three members of the trustees' committee were already in the room. A few minutes later, Leonard Fitch arrived, also carrying a briefcase. "Well, everybody seems to be here," Whitburn said, starting toward his chair behind the desk. "We might as well get this started." "Yes. If you'll excuse me, Doctor." Dacre stepped in front of him and sat down at the desk. "I've been selected as chairman of this committee; I believe I'm presiding here. | |||||
| And the camps we made when their strength outplayed and the day was pinched and wan; And oh, the joy of that blessed halt, and how I did dread the dawn; And how I hated the weary men who rose and dragged me on. And oh, how I begged to rest, to rest--the snow was so sweet a shroud; And oh, how I cried when they urged me on, cried and cursed them aloud; Yet on they strained, all racked and pained, and sorely their backs were bowed. | |||||
| I peddle around here a lot. My father's dead, I haven't got any relatives except a sick aunt that I go to see once in a while, and I'm in business for myself." "You are quite a little soldier," complimented Betty, as she got out the bandages and salve. "You are very brave." "Oh, I haven't got any kick coming," he answered, with a laugh. "Of course, this cut foot will make me travel slow for a while, and I can't get to all my customers on time. But I guess they'll save their trade for me--the regulars will. "I might be worse off," the lad continued, after a pause. "I might be in as bad a hole as that fellow I saw on the train not long ago." "How was that?" asked Betty, more for the sake of saying something rather than because she was interested. | |||||
| These two cases are given simply because they have been fixed in my mind; they are but types of what is going on everywhere throughout civilization: the world is everywhere growing uglier and more commonplace, in spite of the conscious and very strenuous efforts of a small group of people towards the revival of art, which are so obviously out of joint with the tendency of the age that, while the uncultivated have not even heard of them, the mass of the cultivated look upon them as a joke, and even that they are now beginning to get tired of. Now, if it be true, as I have asserted, that genuine art is an unmixed blessing to the world, | |||||
| This is only a line to tell you all how much father loves you. The Pawnee Indian drew you the picture of the little dog, which runs everywhere round the ship, and now and then howls a little when the band plays. Near Santiago, May 20, 1898. DARLING ETHEL: I loved your little letter. Here there are lots of funny little lizards that run about in the dusty roads very fast, and then stand still with their heads up. Beautiful red cardinal birds and tanagers flit about in the woods, and the flowers are lovely. But you never saw such dust. | |||||
| "Every single day, mother!" "Every single thing that happens." "A fat letter every morning," they promised in chorus. "If there is any real trouble remember to telegraph your Uncle Allan--did you write down his address, 11 Broad Street, New York? Don't bother him about little things, for he is not well, you know." Gilbert displayed a note-book filled with memoranda and addresses. "And in any small difficulty send for Cousin Ann," Mrs. Carey went on. "The mere thought of her coming will make me toe the mark, I can tell you that!" was Gilbert's rejoinder. "Better than any ogre or bug-a-boo, Cousin Ann is, even for Peter!" said Nancy. |
@article{peng2025voicestar,
author = {Peng, Puyuan and Li, Shang-Wen and Huang, Po-Yao and Mohamed, Abdelrahman and Harwath, David},
title = {VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation},
journal = {arxiv},
year = {2025},
}