forums.ps2dev.org Forum Index forums.ps2dev.org
Homebrew PS2, PSP & PS3 Development Discussions
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Investigation of the Unicode version of sceIoDread

 
Post new topic   Reply to topic    forums.ps2dev.org Forum Index -> PSP Development
View previous topic :: View next topic  
Author Message
Tong



Joined: 20 Apr 2007
Posts: 10

PostPosted: Wed Jan 28, 2009 1:11 pm    Post subject: Investigation of the Unicode version of sceIoDread Reply with quote

I am working on a file browser that lists the files in the memory stick. I implement using sceIoDopen followed by sceIoDread. I hope to support showing filenames of different languages, including Chinese, Korean and some other western languages apart from English.

Unfortunately, sceIoDread has serious problems:

1. It returns filenames encoded in Shift-JIS (SJIS) which is used for the Japanese only. The function fails to handle filenames with characters that can't be mapped by SJIS. This issue was discussed at http://forums.ps2dev.org/viewtopic.php?t=8949&highlight=sceiodread

2. The broken filename obtained from sceIoDread CANNOT be used to open file with sceIoOpen. (Is it the reason for being unable to load ISOs with non-ascii/non-japanese filenames in CFW?)


There are solutions to this problem:

1. Write a custom fat driver to read Unicode filenames. This approach is used by pmplayer-advance (fat.c). However, it is slow and has low compatibility with other homebrews.

2. Use the idea from alexp (http://forums.ps2dev.org/viewtopic.php?t=8590) who use sceIoDevctl and sceIoIoctl to read Unicode filenames. But this solution is valid only for old firmware, I have confirmed it can't work on recent ones (4.xx, 5.xx).

I am crazy about this issue as I see the vsh CAN read and display these non-ascii / non-japanese filenames properly. I am sure they use sceIoDread too. I have done some sample tests to prove that Unicode version of sceIoDread really exists: (test platform: CFW 5.00 M33-6)

1. Put a image named "Hello 你好嗎" to the PICTURE folder of the memory stick. It is obvious that its name is displayed properly in vsh.


2. Open PSPLINK 3.0OE, do
Code:
reset vsh
cd ms0:/PICTURE
ls


3. Load start flash0:/vsh/module/vshmain.prx to open vsh.

4. Do ls again in PSPLINK, note that the filename of the image is different!



I do some memory dump and found that the filename in step 2 is "Hello ■
好■.bmp" in SJIS. However, the one in step 4 is "Hello 你好嗎.bmp" in UTF-8!

I suspect that the vsh_module(vshmain.prx) has done something to the fatms driver to enable Unicode sceIoDread.

Can anyone help me investigate how it can do so?
Secondly, what we can do in our homebrew games to enable Unicode sceIoDread?

Btw, I also found that vsh directly pass the utf-8 filenames to sceIoOpen and sceIoDopen, are these IO functions support utf-8 too?

thanks
Back to top
View user's profile Send private message
TyRaNiD



Joined: 18 Jan 2004
Posts: 918

PostPosted: Wed Jan 28, 2009 5:37 pm    Post subject: Reply with quote

What does psplink do if you list a UTF8 filename on host? Although you might not be able to do it on Windows (you probably need Linux for that kinda trick as I make not effort to encode the filenames appropriately). That would at least narrow down to whether it is something in the guts of sceIoDread itself or in the actual ms driver.

If it is in the driver itself then about the only thing I can suggest is to look at what IOCTL/DEVCTL calls are made to it by hooking those in psplink and dumping the output, see if you can find one which is being called to set the encoding.
Back to top
View user's profile Send private message
Tong



Joined: 20 Apr 2007
Posts: 10

PostPosted: Thu Jan 29, 2009 12:42 pm    Post subject: Reply with quote

Thanks for your reply.

Quote:
What does psplink do if you list a UTF8 filename on host? Although you might not be able to do it on Windows (you probably need Linux for that kinda trick as I make not effort to encode the filenames appropriately). That would at least narrow down to whether it is something in the guts of sceIoDread itself or in the actual ms driver.


Actaully, I apihooks sceIoDread and then use the SAVEMEM command of psplink to get the SceIoDirent structure returned by the function. I open the mem file with firefox and adjust the encoding settings to see what happen. This is not convenient but at least it works fine.

Quote:
If it is in the driver itself then about the only thing I can suggest is to look at what IOCTL/DEVCTL calls are made to it by hooking those in psplink and dumping the output, see if you can find one which is being called to set the encoding.


Yes, I have thought of that too. But I found that sceIoDread changes to unicode BEFORE any calls to sceIoDevctl or sceIoIoctl. Perhaps these 2 functions are not related to this issue.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.ps2dev.org Forum Index -> PSP Development All times are GMT + 10 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group