forums.ps2dev.org Forum Index forums.ps2dev.org
Homebrew PS2, PSP & PS3 Development Discussions
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Porting to PSP of Speech recognition Pocket Sphinx

 
Post new topic   Reply to topic    forums.ps2dev.org Forum Index -> PSP Development
View previous topic :: View next topic  
Author Message
mypspdev



Joined: 11 Jul 2007
Posts: 178
PostPosted: Thu Jan 03, 2008 1:26 am    Post subject: Porting to PSP of Speech recognition Pocket Sphinx Reply with quote
Hi! Happy 2008 to all!
Here is source code and documentation: http://www.speech.cs.cmu.edu/pocketsphinx/.

I've completed the porting to PSP, it picks up audio input and propose some sentences according to best fit of utterance.
It's fast and nice.
I'm experiencing some crashes at the following procedure where is something that I do not understand.

I've a pointer to unsigned char:
Code:
unsigned char *pid_cw0 ...


Then it's firstly used:
Code:
pid_cw0 = (s->OPDF_8B[j][s->f[j][0].codeword]);


But when used here:
Code:
tmp1 = pid_cw0[n] + w0;

the program on PSP crashes. Something is wrong with [n] position reading.

I've substituted pid_cw0[n] with forced values: the program flows over to end.
Something is wrong reading pid_cw0[n]. and I'am not able to understand what's the meaning of reading this pointed value at position [n].

Is there missing any alloc of array? or something similar? Any initialization of [n] positions is required?
The original source code is fully and succesfully compiled both for PSP and for PC under Windows, but when running on PSP it crashes at that instruction.

Has anybody any suggestion on investigating or by-pass or complete this original instruction set?
Thanks a lot, here is source code of the procedure, then the structure of s2_Semi_mgau_t.

Code:
static int32
get_scores4_8b(s2_semi_mgau_t * s)
{
    register int32 j, n, k;
    int32 tmp1, tmp2;
    unsigned char *pid_cw0, *pid_cw1, *pid_cw2, *pid_cw3;
    int32 w0, w1, w2, w3;       /* weights */

    memset(senone_scores, 0, s->CdWdPDFMod * sizeof(*senone_scores));
    for (j = 0; j < s->n_feat; j++) {

        /* ptrs to senone prob ids */
        pid_cw0 = (s->OPDF_8B[j][s->f[j][0].codeword]);
        pid_cw1 = (s->OPDF_8B[j][s->f[j][1].codeword]);
        pid_cw2 = (s->OPDF_8B[j][s->f[j][2].codeword]);
        pid_cw3 = (s->OPDF_8B[j][s->f[j][3].codeword]);

        w0 = s->f[j][0].val.score;
        w1 = s->f[j][1].val.score;
        w2 = s->f[j][2].val.score;
        w3 = s->f[j][3].val.score;

        /* Floor w0..w3 to 256<<10 - 162k */
        if (w3 < -99000)
            w3 = -99000;
        if (w2 < -99000)
            w2 = -99000;
        if (w1 < -99000)
            w1 = -99000;
        if (w0 < -99000)
            w0 = -99000;        /* Condition should never be TRUE */

        /* Quantize */
        w3 = (511 - w3) >> 10;
        w2 = (511 - w2) >> 10;
        w1 = (511 - w1) >> 10;
        w0 = (511 - w0) >> 10;

        for (k = 0; k < n_senone_active; k++) {
       n = senone_active[k];

//#### HERE IS CRASHING!
            tmp1 = pid_cw0[n] + w0;
//#### HERE IS CRASHING!

            tmp2 = pid_cw1[n] + w1;
            tmp1 = LOG_ADD(tmp1, tmp2);
            tmp2 = pid_cw2[n] + w2;
            tmp1 = LOG_ADD(tmp1, tmp2);
            tmp2 = pid_cw3[n] + w3;
            tmp1 = LOG_ADD(tmp1, tmp2);

            senone_scores[n] -= tmp1 << 10;
        }
    }
    return 0;
}


Code:
typedef struct {
    union {
        int32   score;
        int32   dist;   /* distance to next closest vector */
    } val;
    int32 codeword;      /* codeword (vector index) */
} vqFeature_t;
typedef vqFeature_t *vqFrame_t;

typedef struct s2_semi_mgau_s s2_semi_mgau_t;
struct s2_semi_mgau_s {
    int32   **dets;   /* det values foreach feature */
    mean_t  **means;   /* mean vectors foreach feature */
    var_t   **vars;   /* var vectors foreach feature */

    unsigned char ***OPDF_8B; /* mixture weights */

    int32 n_feat;   /* Number of feature streams */
    int32 *veclen;   /* Length of feature streams */
    int32 n_density;   /* Number of mixtures per codebook */

    int32 topN;      /* Number of top densities to compute (<S2_MAX_TOPN) */
    int32 CdWdPDFMod;   /* Legacy thing, actually means number of mixw */

    kd_tree_t **kdtrees;
    uint32 n_kdtrees;
    uint32 kd_maxdepth;
    int32 kd_maxbbi;

    int32 num_frames;
    int32 ds_ratio;

    int32 use_mmap; /**< Are mixture weights mmap()ed? */

    /* Top-N scores and codewords from current, last frame. */
    vqFeature_t **f, **lastf;
    int32 *score_tmp;
};
Back to top
View user's profile Send private message
Art



Joined: 09 Nov 2005
Posts: 647
PostPosted: Thu Jan 03, 2008 10:44 am    Post subject: Reply with quote
If I read correctly you've declared pid_cw0 as a single character variable,
but then you treat it as an array with the line you say is crashing it.
As I see, unless the value of n is zero, the line you say is crashing it
would change the value of some other byte in memory.
_________________
If not actually, then potentially.
Back to top
View user's profile Send private message
jimparis



Joined: 10 Jun 2005
Posts: 1179
Location: Boston
PostPosted: Thu Jan 03, 2008 2:39 pm    Post subject: Reply with quote
There could be lots of reasons, that code is complicated.
For example, if OPDF_8B is full of junk data than the error makes sense. Printing out the values of all relevant variables (pid_cw0, OPDF_8B, n, etc) might help pinpoint the error.
Back to top
View user's profile Send private message
mypspdev



Joined: 11 Jul 2007
Posts: 178
PostPosted: Thu Jan 03, 2008 6:42 pm    Post subject: Reply with quote
Thanks for replies and helps.
I've checked that OPDF_8B is ***, a 3d array 4x256x5220.
pid_cwX is pointing to the array 4x256 and should have 5220 elements.
The crash happens also with n=170, so <5220.
The memory space for OPDF_8B is allocated correctely in a previous function without errors: I am excluding memory space saturation on PSP, but I'm still investigating.
Now I'll dump the full 3d array values of OPDF_8B.
For who interested I'll explain how to port Pocket Sphinx to PSP: 4 libraries must be compiled (from SphinxBase), then the running programs of Pocketsphinx in the main.c.
Back to top
View user's profile Send private message
mypspdev



Joined: 11 Jul 2007
Posts: 178
PostPosted: Thu Jan 03, 2008 7:28 pm    Post subject: {SOLVED}Porting to PSP of Speech recognition Pocket Sphinx Reply with quote
mistake... sorry

Last edited by mypspdev on Thu Jan 03, 2008 7:31 pm; edited 1 time in total
Back to top
View user's profile Send private message
mypspdev



Joined: 11 Jul 2007
Posts: 178
PostPosted: Thu Jan 03, 2008 7:30 pm    Post subject: Re: {SOLVED}Porting to PSP of Speech recognition Pocket Sphi Reply with quote
[SOLVED]
Now PSP has also Speech Recognition by both audio input: GoCam or Socom Microphone!!

I've eliminated the pid_cwX pointers and I've put in place of them directly the array OPDF_8B[i][j][k] element required.

It works nice!
A new Module is added to MyPSP Robotics: MyTalk, Voice command recognition!
Back to top
View user's profile Send private message
Art



Joined: 09 Nov 2005
Posts: 647
PostPosted: Sat Jan 05, 2008 12:12 pm    Post subject: Reply with quote
Have you posted a demo anywhere we can try?

Did you user the Genuine Socom headset or DIY mic?

I have never found a DIY mic as clear, or loud as an original PSP one.
I wonder if the difference will cause any issues.
_________________
If not actually, then potentially.
Back to top
View user's profile Send private message
mypspdev



Joined: 11 Jul 2007
Posts: 178
PostPosted: Sun Jan 06, 2008 1:30 am    Post subject: Reply with quote
sorry... mistake, I apologize for double posts.

Last edited by mypspdev on Sun Jan 06, 2008 1:34 am; edited 1 time in total
Back to top
View user's profile Send private message
mypspdev



Joined: 11 Jul 2007
Posts: 178
PostPosted: Sun Jan 06, 2008 1:33 am    Post subject: Reply with quote
mypspdev wrote:
Art wrote:
Have you posted a demo anywhere we can try?

Did you user the Genuine Socom headset or DIY mic?

I have never found a DIY mic as clear, or loud as an original PSP one.
I wonder if the difference will cause any issues.


here is MyPSP v5.0.0 with the last module for voice recognition (english) porting Pocketsphinx :
http://rapidshare.com/files/81029828/MyPSPv5.0.0.rar.html
Sorry, some bugs are rised now on old modules because RAM is heavely occupied.
I'm changing dimensions of thread spaces and buffers and MyPSPv5.1.0 is going to be delivered fully working over all modules again.

MyPSPv5.0.0 is only for Slim, doesn't work on Fat (RAM shortage).

Both microphones of coyrse for MyTAlk: Socom headset and GoCam/Talkman USB mic.

Now MyPSP has integrated:
- MyVoice: Text to speech or Text to Wave (Flite)
- MyTalk: speech recognition (Pocketsphinx)
- MyEye: OpenCV, SIFT, GOCR
- MyVVoIP: Video and Voice over IP
- MyAI: OpenMind Knowledge Data base

Hi Art!
Back to top
View user's profile Send private message
J.F.



Joined: 22 Feb 2004
Posts: 2906
PostPosted: Sun Jan 06, 2008 6:44 am    Post subject: Reply with quote
MyTalk with a SOCOM input blows up at the main utterance loop. The MyAudio recorder works fine, so it's probably not the SOCOM input routines.

This is on a Slim with 3.71 M33-4.
Back to top
View user's profile Send private message AIM Address
mypspdev



Joined: 11 Jul 2007
Posts: 178
PostPosted: Sun Jan 06, 2008 7:21 am    Post subject: Reply with quote
J.F. wrote:
MyTalk with a SOCOM input blows up at the main utterance loop. The MyAudio recorder works fine, so it's probably not the SOCOM input routines.

This is on a Slim with 3.71 M33-4.
you are right I'm sorry! I'll prepare the socom update soon.
Back to top
View user's profile Send private message
mypspdev



Joined: 11 Jul 2007
Posts: 178
PostPosted: Sun Jan 06, 2008 8:20 am    Post subject: Reply with quote
J.F. wrote:
MyTalk with a SOCOM input blows up at the main utterance loop. The MyAudio recorder works fine, so it's probably not the SOCOM input routines.

This is on a Slim with 3.71 M33-4.


@J.F.:
tomorrow I'll correct MyTalk for Socom, I apologize for GoCam activation even if you choose Socom Audio Input.
At present, may be it's better to use MyPSPv5.1.0
http://rapidshare.com/files/81479325/MyPSPv5.1.0.rar.html
or here
http://www.psp-ita.com/?module=news&id=12754&view_reply=1
RAM shortage needs some reduced spaces for threads and buffers, v5.1.0 now should work well also with MyVVoIP module for Video and Voice TRX from PSP-Server to PSP-Client even using Socom or GoCam audio input.
Tomorrow I'll deliver also a MyVVoIP module with full duplex Video and Voice trx-rx between two PSPs.
(in the package some prxs by Sony are not included, please get them)

J.F., do you have the remote-controller for slim? is it the same than RC for Fat, of course with the little plug?

Thanks for interest.
Back to top
View user's profile Send private message
J.F.



Joined: 22 Feb 2004
Posts: 2906
PostPosted: Sun Jan 06, 2008 9:08 am    Post subject: Reply with quote
mypspdev wrote:
J.F. wrote:
MyTalk with a SOCOM input blows up at the main utterance loop. The MyAudio recorder works fine, so it's probably not the SOCOM input routines.

This is on a Slim with 3.71 M33-4.


@J.F.:
tomorrow I'll correct MyTalk for Socom, I apologize for GoCam activation even if you choose Socom Audio Input.
At present, may be it's better to use MyPSPv5.1.0
http://rapidshare.com/files/81479325/MyPSPv5.1.0.rar.html
or here
http://www.psp-ita.com/?module=news&id=12754&view_reply=1
RAM shortage needs some reduced spaces for threads and buffers, v5.1.0 now should work well also with MyVVoIP module for Video and Voice TRX from PSP-Server to PSP-Client even using Socom or GoCam audio input.
Tomorrow I'll deliver also a MyVVoIP module with full duplex Video and Voice trx-rx between two PSPs.
(in the package some prxs by Sony are not included, please get them)

J.F., do you have the remote-controller for slim? is it the same than RC for Fat, of course with the little plug?

Thanks for interest.


Of course I have the remote - you can't use the SOCOM headset mike without it. :D The slim remote is a smidgen different from the phat - it has a different end connector (since the connector on the slim is different because of the video out), and the remote itself is a smidgen different. Like I said, your voice recording routine worked fine, so the SOCOM is working, and the code for that part is working fine.
Back to top
View user's profile Send private message AIM Address
mypspdev



Joined: 11 Jul 2007
Posts: 178
PostPosted: Sun Jan 06, 2008 9:13 am    Post subject: Reply with quote
J.F wrote:
Like I said, your voice recording routine worked fine, so the SOCOM is working, and the code for that part is working fine.


For using Socom or GoCam, it's simple:

Code:
int pspAudioInput(s32 length, s32 sample_rate, void *buf) {
   if (!audio_input_ready)
      return -1;
      
   if (MyMic==PSP_GOCAM_MIC)
   {   
      sceUsbCamReadMic(buf, length);
   }
   if (MyMic==PSP_SOCOM_MIC)
   {
            sceAudioInput(length, sample_rate, buf);
   }
   return 0;
}


Code:
int pspAudioInputBlocking(s32 length, s32 sample_rate, void *buf) {
   if (!audio_input_ready)
      return -1;
   if (MyMic==PSP_GOCAM_MIC)
   {   
      sceUsbCamReadMicBlocking(buf, length);
   }
   if (MyMic==PSP_SOCOM_MIC)
   {
         sceAudioInputBlocking(length, sample_rate, buf);
   }
   return 0;
}


I've not tested well Socom on Slim because I have remote control only for fat... now I'm going to separate MyTalk and the reduced stand alone module should work also on fat.
Tomorrow I'll complete the input from Socom, too.
Back to top
View user's profile Send private message
mypspdev



Joined: 11 Jul 2007
Posts: 178
PostPosted: Sun Jan 06, 2008 9:27 am    Post subject: Reply with quote
sorry mistake...

Last edited by mypspdev on Sun Jan 06, 2008 9:36 am; edited 1 time in total
Back to top
View user's profile Send private message
mypspdev



Joined: 11 Jul 2007
Posts: 178
PostPosted: Sun Jan 06, 2008 9:36 am    Post subject: Reply with quote
[quote="mypspdev"]@J.F.:
for Socom Audio Input to MyTalk, please would you try on Slim to use this eboot.pbp in place of previous one on the same folder MyPSPv5.1.0 ?
I've added the if (MyMic==PSP_SOCOM_MIC)...
let me know, thanks

EDIT:
Here new eboot for Socom Audio Input for MyTalk:
http://rapidshare.com/files/81579280/EBOOT.rar.html
Back to top
View user's profile Send private message
Art



Joined: 09 Nov 2005
Posts: 647
PostPosted: Thu Jan 24, 2008 11:01 pm    Post subject: Reply with quote
I was wondering the difference for a DIY mic compared to a genuine one
(both are Socom mics). I guess I'll try, but I can't compare to a genuine
Sony Socom mic anymore.
_________________
If not actually, then potentially.
Back to top
View user's profile Send private message
jean



Joined: 05 Jan 2008
Posts: 489
PostPosted: Sun Jan 27, 2008 2:20 am    Post subject: Reply with quote
Just to reply @Art... i've sectioned PSP's remote :) and even if it's all thight and integrate SMD technology, i can guess that the fourth ring of the audio connector (that SOCOM mic uses) doesn't go directly to the "serial" connector, but is preamplified (we could do as well in a DIY mic with a simple single-transistor circuit powered by the SIO +/- pins). Moreover, this is strictly necessary if your microphonic capsule is a capacitive (and not resistive) one, as those you can find in mobile GSM phones'. By now, i'm using a nokia mic soldered directly into the PSP remote, and it's working fine, but if you want to be independant from remote controller, then you should try different mics and (if you want it loud) some simple preamplifiers. PM if you want some schematics.

jean
Back to top
View user's profile Send private message
Art



Joined: 09 Nov 2005
Posts: 647
PostPosted: Fri May 16, 2008 4:56 pm    Post subject: Reply with quote
Sorry, I missed that one at the time you posted it jean.

Even when I used a DIY headset with 3.5mm 4 conductor plug via the headphone remote,
it still
sounded like crap, and furthermore, I destroyed my first Socom headset
to try to get to the bottom of it, and the mic works just as well connected straight to the headphone port on the PSP.

The mic itself is just a little electret mic with one cap surface mounted on a tiny PCB.
It didn't look special.

Now I'm on my second copy of Socom Fire Team Bravo :rolleyes: :D

Looks like the answer to get the Talkman mic going with Audio Mechanica might be a few posts above,
if the Talkman mic works as easy as the GoCam one.
I don't know if anyone can use it yet. Is it any different?
_________________
If not actually, then potentially.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.ps2dev.org Forum Index -> PSP Development All times are GMT + 10 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group