Keyboards and AI

[RoyB]

Youtube is awash now with AI generated music - from AI generated compositions and arrangements, AI generated cover songs using the AI generated voice of a different singer, to AI generated voices of famous singers used as a vocal to own-compositions. And some of them are really good.

It is all very impressive and scary at the same time.

There are already AI plugins available for DAWs. So, how long will it be before AI functionality and/or integration finds its way into synthesizer and/or arranger keyboards?

Presumably, it could be possible for keyboards to be able to embrace AI in someway to generate loops, styles, accompaniments and even full arrangements, to add AI vocals to keyboard playing, and to help in compositions.

Is this the future for the next generation of workstations and/or arrangers?

Regards

Roy

[Gleston]

If the AI comes to help us play ours instruments and not substitute us , then it will be welcome.
As long as human playing keyboard is the focus, any advancement will be useful.

Whats is the point in use AI to completely play the song? If music and instruments were create for us to play?

I´m not scare of AI , I prefer to play myself , with some interaction with any AI software will be great , but I´m always in command of my playing and pleasure.

Here´s an idea, we can use AI to introduce a new feature in the arrangers : today we have the factory styles well made  by professionals, we can transform some MIDI files in arrangers style, we can mix the existing style and create another one (STYLE CREATOR) and now maybe we can use the AI to create a new style for US to play along (STYLE GENERATOR??)

We can set parameters for this style , include witch style we want ( pop, rock, ballad, etc.) and let the keyboard generates a new one and then keep changing until WE are satisfied and use it for OUR pleasure.

Focus on humans after all.  The main goal of an arranger keyboard is for SOMEONE to play it.

[J. Larry]

While playing an arranger, it would be nice to be able to hit a button and get an authentic-sounding solo played on any selected instrument voice----played in the style of the song, following the chord changes, with no glitches.  That's asking a lot, for sure.  However, Band In A Box has real track solos available to augment the songs created.

[pjd]

As the manual sez, "AI Full Keyboard may not always produce appropriate accompaniment."

I would love to see continual improvements in this area. Yamaha does a ton of research on music analysis -- seems like a good area for improvement. Should improve Chord Tracker's recognition, too.

Some form of auto-accompaniment is trickling into the ordinary digital piano product line. The P-515 provides drum plus bass accompaniment. No mention of how the P-515 decides on the bass notes/accompaniment or chord recognition other than the cautionary statement "Some arrangements may not be suitable for use with this feature. 9th, 11th and 13th chords cannot be played."

They got some work to do -- pj

P.S. Yep, I know about the arranger features in CVP, before anybody mentions it. 

[Mark]

What a great topic to discuss! Unfortunately, it looks like the AI ideas haven't found their way into the arranger's world... Well, maybe in a decade or two, who knows? Nevertheless, here are some ideas:

1. WAVE / MP3 files processing with AI:
Assuming AI can "listen" to the file and "interpret it" as humans do AI could:
- Suggest a tempo of the composition
- Suggest Scale(s)
- Recognize Chords
- Extract Melody / Bass Line / Drum patterns and whatnot based on our preferences (if humans can make it "by ear" maybe AI also can). Convert it to midi with the "closest" voice and provide a staff notation/part that can be imported into the style engine, sequencer, etc.

2. Generation of Musical Ideas:
Based on the style and played composition AI could "support" the player in real-time by generating the whole phrases (pretty much like multi pads do) - it could be melodic parts, or maybe some minor things (percussive fill-ins, bass moves, etc) to make a performance more "interesting".

3. Midi file Intelligent Analysis. AI could analyze a midi song and suggest converting it into style by "extracting" parts that suit variations, fill-ins, etc.

4. Integration with Style Creation Engine. Here AI can be used in many different ways as well:
- Intelligent assistance in choosing the parameters (NTT / NTR): a player could play the pattern exactly as he/she thinks it should be and the AI will analyze it and convert it into the Style Format that the keyboard will understand and re-apply in real-time, so that there won't be even a need to know about all these NTR / NTT parameters and note restrictions (like "you're allowed to use only note C, D, E, but not F and G# ) It could be also an interactive process: the player plays in Major scale, the AI generates and suggests for auditioning how it will sound like in a minor scale, and when the user "confirms" that it sounds like it should, - convert (again automatically by AI) into internal style structure form.
- Support a "part" generator. For example, I have drums and bass, so the style has its "basic" groove and feeling. But I'm not a skillful guitar player, so I could ask AI to "generate" a guitar part
for me that will "feel" natural with the style.
- Mixing advice: we could get suggestions from AI how to setup EQa nd Mixing properly to get a better sound.

5. Suggest songs to the style and the other way around:
- When we hear a song that we would like to play we have to find an "appropriate" style, set up the tempo, maybe turn off some parts, etc. AI could analyze all the styles that we have (both factory and user styles) and suggest the best style for the song.

6. Intelligent mistakes correction: Hear our play (like a piano teacher does) and provide a feedback on what should be improved

7. An ultimate task for AI: Create a Midi / Style out of any arbitrary WAV / MP3 song automatically )))

[Duffy]

Good points Mark,
That all sounds easily feasible to me and I would expect Ai to be one of the biggest aids to amateur / home musicians for a long time, certainly since rhythm players first appeared.
Sounds  as if it could be the thing that replaces midi, although I rarely use midi files, except for fun, and certainly never when playing in front of others.
It should be a breeze for something like that to write new styles for us.

[Gleston]

Dear Mark

Just to go a litlle bit further with this subject and thinking that music is for HUMANS to play, there´s goes my opinion:

1. WAVE / MP3 files processing with AI:
Assuming AI can "listen" to the file and "interpret it" as humans do AI could:
- Suggest a tempo of the composition
- Suggest Scale(s)
- Recognize Chords
- Extract Melody / Bass Line / Drum patterns and whatnot based on our preferences (if humans can make it "by ear" maybe AI also can). Convert it to midi with the "closest" voice and provide a staff notation/part that can be imported into the style engine, sequencer, etc.



It´s part of my process of finally playing a music, try to figure out all this features of a song. In that part I don´t want an AI to help me.





2. Generation of Musical Ideas:
Based on the style and played composition AI could "support" the player in real-time by generating the whole phrases (pretty much like multi pads do) - it could be melodic parts, or maybe some minor things (percussive fill-ins, bass moves, etc) to make a performance more "interesting".

This is good as long as we use it only as a suggestion. Beware to not use it as a main player ( the AI) . Always it is US as HUMANS that should be in the command of the song


3. Midi file Intelligent Analysis. AI could analyze a midi song and suggest converting it into style by "extracting" parts that suit variations, fill-ins, etc.


Personally I love to do this by hand, using JOSOFT MIDI TO STYLE software.  In the process of transforming a MIDI file into a STYLE makes me learning the song better, the parts of the song, and when finally it´s ready, it became easy for me to play it in the keyboard.



4. Integration with Style Creation Engine. Here AI can be used in many different ways as well:
- Intelligent assistance in choosing the parameters (NTT / NTR): a player could play the pattern exactly as he/she thinks it should be and the AI will analyze it and convert it into the Style Format that the keyboard will understand and re-apply in real-time, so that there won't be even a need to know about all these NTR / NTT parameters and note restrictions (like "you're allowed to use only note C, D, E, but not F and G# ) It could be also an interactive process: the player plays in Major scale, the AI generates and suggests for auditioning how it will sound like in a minor scale, and when the user "confirms" that it sounds like it should, - convert (again automatically by AI) into internal style structure form.
- Support a "part" generator. For example, I have drums and bass, so the style has its "basic" groove and feeling. But I'm not a skillful guitar player, so I could ask AI to "generate" a guitar part
for me that will "feel" natural with the style.
- Mixing advice: we could get suggestions from AI how to setup EQa nd Mixing properly to get a better sound.


That´s is part of what I´m talking about.






5. Suggest songs to the style and the other way around:
- When we hear a song that we would like to play we have to find an "appropriate" style, set up the tempo, maybe turn off some parts, etc. AI could analyze all the styles that we have (both factory and user styles) and suggest the best style for the song.


I love do that myself, listen to so many styles that we can download from this forum and experiment with it until we find one that we can use, even if we have to make few adjustements or it comes completely ready for the song we are trying to play. It is also in the process of learning a song. No AI needed.





6. Intelligent mistakes correction: Hear our play (like a piano teacher does) and provide a feedback on what should be improved


I think this could help, but you also could record yourself playing, and listen after a few days so you can hear where you can improve. Try and error never fails!!!

By the way, listen to what Pedro Eleuterio is doing! It´s a member of this forum and I think he does a very good job all by himself (creating style and playing it) !!




Your topic nr 7 can  already be done. Use an AI software that separate the stems of an mp3 song (RIP-X, MOISES AI, RX7,etc) and then YOU  transform it to MIDI Style. Whats is the fun in doing it automatically!!!

[EileenL]

We do have AI on our keyboards now or as it is called Artificial intelligence which lets you play slashed chords with just two fingers. On board bass is a little hard for people who don't know all the inversions of every chord.
To use it on all the things mentioned above to me is way over the top. What does it   Leave for the player to do.

[Mark]

Oh, I took this topic from a computer perspective thinking about where AI can be applied for arrangers.
I should have put a disclaimer that all those items could be implemented as "suggestions" otherwise we'll miss the player's "soul" in these automated processes.

In addition, we all have different skills and abilities, more "pro-level" people can come up with decent arrangements prepared by themselves, while others are hobbyists who could benefit a lot from various features provided by AI. It's pretty much like a "transposer" for me if you wish - some people prefer not to use it because they play in different scales, but for others, it can be a savior

Of course, it's all theoretical discussion, Yamaha is very "conservative" in features for their new instruments, AFAIK, so I'll be really surprised if any feature with AI will be implemented in their arrangers within the next 5-10 years or something

@EileenL - I'm not sure it qualifies as "AI" meant in this topic, I might be wrong, but it might be a well-defined (deterministic) algorithm that knows exactly what to generate based on the simplified form of the chord. Engineers have defined a series of rules "how" the system should react given the last played notes. AI on the other hand is more about "I have no idea how to solve (generate/play/determine scale or tempo, etc)" - and asking for the help of an "intelligent" musician who "lives" inside the keyboard. The point is that it's not a preprogrammed-in-advance algorithm (not even randomized).

[pjd]

Some form of auto-accompaniment is trickling into the ordinary digital piano product line. The P-515 provides drum plus bass accompaniment. No mention of how the P-515 decides on the bass notes/accompaniment or chord recognition other than the cautionary statement "Some arrangements may not be suitable for use with this feature. 9th, 11th and 13th chords cannot be played."

Well, I might have answered my own question as far as the P-515 drum/bass accompaniment is concerned. Circling back to the old Tyros2 Owner's Manual, it has the same disclaimers as the P-515. So, I would conclude that the P-515 (and its digital piano relatives) use "AI Full Keyboard" all the time for chord recognition.

Given that Yamaha's "AI" chord fingering has been around for such a long time, maybe Yamaha should revisit? Looking at their patents, I know they investigating features which use contemporary AI technology such as machine learning.

I think ensemble real-time voice assignment could use a lift from AI technology. I wish playing ensemble parts would be as intuitive as Super Art -- and I understand the theory, etc. about multi-part orchestration!   

All the best -- pj

[pjd]

Oh, yeah, Yamaha keep dropping important information about certain features. When in doubt, read old manuals.

I learned a lot about XG effects architecture from old MU series manuals. Yamaha seem to tire with certain technology ("old hat") and drop useful information from manuals. Oh, well.

-- pj

[RoyB]

Many existing 'Smart' technologies, such as chord and bass recognition, whilst requiring some level of 'intelligent' interpretation, don't really meet the modern concept of AI, which involves detailed analysis, the ability to learn, and to interprete, adapt and implement.

Perhaps an example of AI for a keyboard could be this. The AI keyboard constantly analyses all your playing - it analyses what you're playing, how you are playing it, the keys you tend to play most in, the kind of styles you tend to use most, etc. So, after a while, it might learn (amongst a whole lot of other things) that you tend to play mostly in the key of G and that you tend to play a lot of big band music in a certain way. So, when you present it with a midi file of a song that you have obtained from elsewhere, which for example may be an orchestral piece, the AI keyboard can then take that orchestral midi song and present it in the key (G) and in the style (big band music) that you prefer to play, as well as being able to create a suitable 'big band' style for that song based on how to like to play big band music and the big band styles that it knows you like to play.

I would guess that how far keyboard technology can implement AI on-board would only be limited by the imagination of keyboard designers, as well as how much processing power manufacturers can build into keyboards at a realistic price.  It could be that some of the AI processing would need to be done on a PC, and that the end result would need to be transferred to the keyboard in some form.

Like it or not, AI is here to stay, and more and more technological items are going to be adopting it in some form.

Roy

[Divemaster]

I console myself with the fact that we have now probably reached very close to the pinnacle of keyboard instruments. They are rewarding to play, take quite a bit of learning, and produce really good music.

But for me, the emphasis is on Playing.... Not constantly having to program it and faff around with it like a computer. I think once it starts to virtually play itself or the player has everything so ridiculously easy to tell it to do, you might as well buy an MP3 player. There's no skill left, no challenge, no learning to actually play the instrument any more.
I certainly wouldn't spend /waste money on that concept.

Keith

[EileenL]

Hi Divemaster,
  I agree with you one hundred percent. Surely we buy keyboards to play them and to learn how to play to the best of our ability and get great enjoyment from it.
  The way things are going there will be no need for live musicians in the future as it will all be done by robots.

[JohnS (Ugawoga)]

Rick Springfield ------ "We All Need The Human Touch"!!