Genos internal memory for expansion

[pjd]

Hi --

I hope we can shift the discussion to this thread as the technical details do not interest some Forum members.

I posted two new articles on sandsoftwaresound.net which summarize the main CPU and the SWP70 tone generators. The articles include cleaned up diagrams from the Yamaha Service Manual to make sure that the info is correct.

All the best -- pj

http://sandsoftwaresound.net/yamaha-genos-main-cpu/
http://sandsoftwaresound.net/yamaha-genos-tone-generation/

[pjd]

There are a few points in the V2.0 thread which I was hoping to clarify.

First, the Yamaha tone generation architecture uses proprietary integrated circuits (SWP70) for tone generation. This is not the same architecture as PC-based software that streams samples from the file system, performing synthesis on the main CPU. The Genos main CPU performs mainly control and user interface functions.

Yes, there are two SWP70 tone generators. The Master has 2GBytes of wave memory and the Slave has 4GBytes of wave memory. The Master (2GBytes) holds user expansion memory. The Slave (4GBytes) holds the factory preset waveforms. We do not know if there is any spare capacity in the Slave because Yamaha never explicitly discusses such matters.

Please be careful when reading the diagrams. Yamaha often states memory size in BITS not bytes. If discussing bytes, be sure to convert from bits.

All the best -- pj

[Ed B]

Thanks for this info pj. Most informative. Looks like everything is unpacked and you are pretty close to being "normal".  Question can you help me understand what you mean by "studio-level audio conversion".
Regards
Ed B

[voodoo]

Yes, there are two SWP70 tone generators. The Master has 2GBytes of wave memory and the Slave has 4GBytes of wave memory. The Master (2GBytes) holds user expansion memory. The Slave (4GBytes) holds the factory preset waveforms. We do not know if there is any spare capacity in the Slave because Yamaha never explicitly discusses such matters.

Hey pj,

first of all, great articles, as always, great pleasure to read.

Do you have any idea, where the 1 GB memory for expansion audio styles have been located?

Uli

[voodoo]

Hey pj, again,

are you shure what we have 2 GB for expansion and 4 GB for preset? In this post

  https://www.psrtutorial.com/forum/index.php/topic,51887.msg407758.html#msg407758

Chris posts the opposite statement. This would make more sense, since this hypothesis leaves space for 3 GB of expansion or 1 GB of audio styles in the 4 GB expansion chunk.

Uli

[pjd]

Thanks for this info pj. Most informative. Looks like everything is unpacked and you are pretty close to being "normal".  Question can you help me understand what you mean by "studio-level audio conversion".

Hi Ed --

I'm thinking about interfaces like the Steinberg UR824, UR28M, UR44 and UR242 with pre-amps, DACs, ADCs and effects. These interfaces are built around the Yamaha SSP2 chip. The SSP2 chip is at the heart of the Montage digital audio system and provides input/output to a DAW like the URs.

We won't be seeing that kind of capability in the current hardware platform.

Hope this clarifies -- pj

[pjd]

Do you have any idea, where the 1 GB memory for expansion audio styles have been located?

Hi Uli --

Thanks!

I believe that audio expansion styles are stored somewhere on the 64GB eMMC bulk storage device. I didn't say, but one of the important duties of the main CPU software is sequencing. With five channels of digital audio from the main CPU to the Master SWP70, the main CPU can sync audio style playback with MIDI, sending the audio portion of the audio style to the SWP70 as serial digital audio.

It's been a while since I've installed an audio style, but I think it's as simple as copying the audio style to the Genos "Internal Memory (User Drive)". No reboot necessary.

Writing waveforms to SWP70 NAND flash is quite slow and is apparently disruptive enough to require a reboot.

Looking to your feedback on this point as it is a deduction, not a published fact.

All the best -- pj

[pjd]

Chris posts the opposite statement. This would make more sense, since this hypothesis leaves space for 3 GB of expansion or 1 GB of audio styles in the 4 GB expansion chunk.

Yep, Chris's post prompt me to return to the Genos Service Manual. I miss having tech writers and proofreaders (!) and was concerned that I had posted wrong information on my site.

The terms "Master" and "Slave" have to do with the communication relationship between the two SWP70s. The names do not imply "preset" or "user."

Rightly or wrongly, I believe that Genos is almost a superset of the Montage waveforms. (Yeah, I know I'm abusing the word "superset" here. :-) ) Montage has 4GBytes of wave memory which Yamaha spec as 5.76GBytes when converted to 16-bit linear format. The S.Art2 voices are memory hogs, so I believe that Genos waveforms require 4GBytes of physical NAND flash. Thus, I believe that preset waveforms are in Slave SWP70 wave memory.

We know about the polyphony split between preset voices and user voices. And, conveniently, the Master SWP has 2GBytes of wave NAND flash. Allocate a little space for bookkeeping and voice meta-data, and the rest is user expansion space.

That's why I'm thinking that the V2.0 increase in expansion space is due to compression. (Bachus concurs.) Korg probably performed this same trick with its recent Pa series updates.

Hope this explanation is solid. Thanks for your feedback.

-- pj

[voodoo]

Hey pj,

Thanks for your reply. Your explanation sounds solid indeed. So we will see what will happen.

Uli

[voodoo]

I found this statement on the Korg web site

The User Sample memory is now more than three times larger! You can now load 1.5 GB of User Samples, which becomes 3 GB when using our fast compression algorithm.

This supports your thought.

Uli

[voodoo]

Hi Uli --

Thanks!

I believe that audio expansion styles are stored somewhere on the 64GB eMMC bulk storage device.

Hey pj,

Now i can confirm this believe. I installed the bonus audio style packs using the PPF packs, and then the available storage on user drive got reduced from 58.04 GB to 57.42 GB. So the 600 MB of audio styles have been stored to the user drive internally, and no space from the Flash ROM has been allocated. (Nevertheless, the keyboard had to be rebooted, since the Flash ROM had been formatted by loading the PPI.)

So there is no hidden space in the Flash ROM that has been used for audio styles and could be released for wave memory. This strengthens the compression theory.

Uli

[pjd]

Hi Uli --

Thanks for the experiment!

-- pj

[EileenL]

Once you re-save these audio styles to USB you no longer need them in the YEM or user section.
  I have mine saved to the 64 Gb USB I have under the keyboard.

[voodoo]

Pj,

I have read your new blog post

  http://sandsoftwaresound.net/yamaha-genos-montage-sample-compression/

about compression. I just want to add, that Nord uses compressed samples for their sample based instruments. And there, the Nord sample manager does compress user samples to an internal format that is loaded into the Nord keyboards. Following this way, the YEM had to be extended by this function. And I never heard about anybody that wanted to steal the algorithm out of the Nord sample manager for any purpose. So I don't understand this fear.

By the way: Nord user samples allow only for one sample per key. There is no velocity split and no layering supported. Of course Nord keyboards have velocity switching, but only for their proprietary piano library, but not for any kind of sample instruments (brass, strings, basses etc.) and not for user samples.  So in this point, the YEM is much more powerful.

Uli

[voodoo]

Pj,

I found another source of information about memory sizes. After loading N27 files for instruments to YEM, there will be an file called instruments.ini in the YEM data folder. Her we can find the following information:

instruments.ini

[3551_XXX]
ModelName=PSR-S970
SerialNumber=XXX
ID=3551_XXX
VoiceWaveRomSize=509
VoiceWaveRomStartOffsetAddress=0
AudioStyleWaveRomSize=129
PackInstallDataIdentification=0
PackInstallDataFormatType=3
UVFBinaryFormatType=0

[4012_XXX]
ModelName=PSR-SX900
SerialNumber=XXX
ID=4012_XXX
VoiceWaveRomSize=1004
VoiceWaveRomStartOffsetAddress=0
AudioStyleWaveRomSize=0
PackInstallDataIdentification=0
PackInstallDataFormatType=3
UVFBinaryFormatType=0

[3650_XXX]
ModelName=Genos
SerialNumber=XXX
ID=3650_XXX
VoiceWaveRomSize=1843
VoiceWaveRomStartOffsetAddress=0
AudioStyleWaveRomSize=1024
PackInstallDataIdentification=0
PackInstallDataFormatType=3
UVFBinaryFormatType=0

Here we have stated, that in Genos we have 1024 MB available in ROM for audio style waves. With newer versions of Firmware, the audio styles are saved to user drive, and there is no limit of 1024 MB. However, here is a hint for a hidden GB that could be used for other things now.

Uli

[pjd]

Hi Uli --

Thanks for the info. I'm keeping an open mind since the V2.0 pre-announcement has no details. Yamaha have definitely surprised me before.  It would be so much easier if Yamaha just published their software architecture. 

Speaking of updates, I'm looking forward to the MODX update. Just a few more days...

Lots to do in the meantime. I'm playing with a new group. About 80% familiar music and about 20% new. The biggest problem so far has been finding space in the choir/music niche. A big group and not much room. I started out with the Reface YC (not too bad), but fitting in the MODX6 was a challenge -- and it's not even that big! Genos would be impossibly big.

All the best -- pj

[voodoo]

Pj,

then you need the Sonogenic SH500. Or an accordion.

Uli

[pjd]

So I don't understand this fear.

Oops, forgot to mention.

I agree that any old compression doesn't sound like a big deal. From what I can gather, Yamaha's approach to compression has some extra smarts.

For example, their compressor first determines if compression is worth it or not. This is probably why some waveforms in Yamaha packs are compressed in LINEAR16_FRAME format instead of WXC.

Since the waveforms are stored in compressed form in the waveform memory and are decompressed on the fly, the decompression algorithm needs to know how to find the location of loop points and such within the compressed data. Early synthesis hardware was quite slow, so it's possible that the tone generator does some additional synthesis processing during decompression. Yamaha AWM combined filtering and effects processing, for example, in order to reduce computation time and latency. I've seen it inferred that processing is also performed during the decompression phase.

I'm still trying to grok why there is LINEAR16_FRAME in addition to good old LINEAR16. I know that frames are part of some time-stretching algorithms, but it doesn't make sense for voice waveforms. This is totally a guess. 

Take care -- pj

[pjd]

Pj,

then you need the Sonogenic SH500. Or an accordion.

Uli

   

After Genos and MODX, there's no turning back!    Maybe a PSS-F30? Hey, we just bought a house...

-- pj

P.S. Up kind of late? I'm still getting used to the new time zone.

[Bachus]

Pj,

I have read your new blog post

  http://sandsoftwaresound.net/yamaha-genos-montage-sample-compression/

about compression. I just want to add, that Nord uses compressed samples for their sample based instruments. And there, the Nord sample manager does compress user samples to an internal format that is loaded into the Nord keyboards. Following this way, the YEM had to be extended by this function. And I never heard about anybody that wanted to steal the algorithm out of the Nord sample manager for any purpose. So I don't understand this fear.

By the way: Nord user samples allow only for one sample per key. There is no velocity split and no layering supported. Of course Nord keyboards have velocity switching, but only for their proprietary piano library, but not for any kind of sample instruments (brass, strings, basses etc.) and not for user samples.  So in this point, the YEM is much more powerful.

Uli

Nord does not pretend to have a sample player... the sinle vellocity samples are part of their synth engine

[JohnS (Ugawoga)]

Once you re-save these audio styles to USB you no longer need them in the YEM or user section.
  I have mine saved to the 64 Gb USB I have under the keyboard.

You can store loads on the USB underneath the keyboard and you could probably put a 128 gig USB in there if needed
.
Eileen is 100% right again
All The Best
John

[JohnS (Ugawoga)]

Pj,

then you need the Sonogenic SH500. Or an accordion.

Uli

Sonogenic is a kids toy!!
Go for an Alesis or Roland!! Then you could be Joe Satriani !!! " Surf with the Alien"!     !!

[voodoo]

You can store loads on the USB underneath the keyboard and you could probably put a 128 gig USB in there if needed
.
Eileen is 100% right again
All The Best
John

Yes, of course we know that audio styles can be loaded from USB. That was not the question. We were discussing, if there is some hidden memory in Genos that has been used for audio styles earlier. Because with Firmware 1.10 audio styles could not be loaded from USB. Understand?

[voodoo]

Sonogenic is a kids toy!!

Did you see the smiley? Of course it is.

[pjd]

Getting back to the original discussion...

I posted two more articles:

http://sandsoftwaresound.net/yamaha-genos-montage-sample-compression/
http://sandsoftwaresound.net/yamaha-montage-internals-revisted/

I'm taking a MODX update holiday today. 

-- pj

[voodoo]

Hey PJ,

from what I have seen, the 3 GB of expansion memory in Genos 2.0 seems to be real 3 GB. Because the 130 MB of the Genos superior pack use only 130 MB of the Genos memory, using YEM 2.6, although it contains compressed voice data. But I haven't tried this with the real thing.....

Uli

[fantomx2019]

Question:
Is it true that the expansion memory has a limited number of formating?
And where I can see how many times Expansion memory was erased?
Thank You.

[pjd]

Hi --

This is a complicated and difficult topic even if we are discussing a single flash memory component. Genos has three different bulk flash memory types: the 64GB eMMC user memory, a 4GB eMMC memory for the OS and 4 x 1GB waveform memory. Each of these memories uses a different flash memory cell design and each memory has a different endurance (number of write/erase cycles).

With respect to eMMC memory, the components have built-in error detection and management: wear leveling, error correction code (ECC), bad block detection and replacement. It takes some digging to find the cell type used in a specific component. The 4GB eMMC device, for example, uses a multi-level cell (MLC) with an endurance in the 30,000 cycle range. Error management is transparent to the OS and I doubt if the OS maintains block by block cycle counts.

Waveform memory uses single-level cell (SLC) NAND flash with a 100,000 write/erase cycle endurance. Error management is the responsibility of the memory controller. I doubt if the SWP70 controller does wear leveling; it's more important to lay down samples in memory sequentially in order to read them at high speed.

Practically speaking, I sleep well at night. I certainly don't write/format waveform memory every day! Even if I did erase waveform memory 10 times a day, it would be 10,000 days until the estimated end of flash life.

I do store new voice info in bulk user memory (64GB) rather frequently when programming new voices. The memory is big and wear leveling is in play. Again, no worries.

However, do I back up all of my work? Yes.

BTW, flash drives are subject to the same endurance issues. Cheap flash drives, especially, may have low endurance. Never trust your flash drives alone. Back up flash drives to more reliable storage, too. Your time is valuable, not just the data.

Hope this helps -- pj

[BogdanH]

Excellent answer pjd -thank you!

Bogdan

[fantomx2019]

Practically speaking, I sleep well at night.
Hope this helps -- pj

It does help.
Thank you.
I can now sleep well too.