forums.ps2dev.org Forum Index forums.ps2dev.org
Homebrew PS2, PSP & PS3 Development Discussions
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The new-style psptoolchain script.
Goto page 1, 2  Next
 
Post new topic   Reply to topic    forums.ps2dev.org Forum Index -> PSP Development
View previous topic :: View next topic  
Author Message
ooPo
Site Admin


Joined: 17 Jan 2004
Posts: 2032
Location: Canada

PostPosted: Thu May 17, 2007 4:19 pm    Post subject: The new-style psptoolchain script. Reply with quote

After releasing the previous toolchain update on my site and watching some of the problems people had, I've updated it:

http://dev.oopo.net/files/psptoolchain-20070517.tar.bz2

There's been no changes to the patches, they're the same as you'd get with the current psptoolchain in the subversion repository.

Now for the why.

1) The toolchain script has grown annoying to manage.
2) There's a lot of random stuff added to the script instead of doing things like properly merging patches.
3) Checking dependencies before running is very important - adding more should be easy to do.

So, here's the new-style script:

Code:
#!/bin/sh
# toolchain.sh by Dan Peori (danpeori@oopo.net)

 ## Create the build directory.
 mkdir -p `dirname $0`/build || { echo "ERROR: Could not create the build directory."; exit 1; }

 ## Enter the build directory.
 cd `dirname $0`/build || { echo "ERROR: Could not enter the build directory."; exit 1; }

 ## Run the depend scripts.
 for SCRIPT in `ls ../depends/*.sh | sort`; do $SCRIPT || { echo "$SCRIPT: Failed."; exit 1; } done

 ## Run the build scripts.
 for SCRIPT in `ls ../scripts/*.sh | sort`; do $SCRIPT || { echo "$SCRIPT: Failed."; exit 1; } done

All it does is create a build directory, run all the scripts in the 'depends' directory, and then all the scripts in the 'scripts' directory.

Adding a new dependency check is as simple as adding a new script in the 'depends' directory. Here's what we currently have:

Code:
psptoolchain/depends/check-autoconf.sh
psptoolchain/depends/check-automake.sh
psptoolchain/depends/check-bison.sh
psptoolchain/depends/check-flex.sh
psptoolchain/depends/check-gcc.sh
psptoolchain/depends/check-make.sh
psptoolchain/depends/check-ncurses.sh
psptoolchain/depends/check-patch.sh
psptoolchain/depends/check-pspdev.sh
psptoolchain/depends/check-subversion.sh
psptoolchain/depends/check-texinfo.sh
psptoolchain/depends/check-wget.sh

Here's what one of these scripts look like:

Code:
#!/bin/sh
# check-make.sh by Dan Peori (danpeori@oopo.net)

 ## Check for make.
 make -v 1> /dev/null || { echo "ERROR: Install make before continuing."; exit 1; }

Simple, yes? That's the point.

Build scripts are similar, but are numbered so that they get executed in the correct order:

Code:
psptoolchain/scripts/001-binutils-2.16.1.sh
psptoolchain/scripts/002-gcc-4.1.0-stage1.sh
psptoolchain/scripts/003-pspsdk-stage1.sh
psptoolchain/scripts/004-newlib-1.15.0.sh
psptoolchain/scripts/005-gcc-4.1.0-stage2.sh
psptoolchain/scripts/006-pspsdk-stage2.sh
psptoolchain/scripts/007-gdb-6.4.sh
psptoolchain/scripts/008-insight-6.4.sh

Here's what one looks like on the inside:

Code:
#!/bin/sh
# binutils-2.16.1.sh by Dan Peori (danpeori@oopo.net)

 ## Download the source code.
 if test ! -f "binutils-2.16.1.tar.bz2"; then
  wget ftp://ftp.gnu.org/pub/gnu/binutils/binutils-2.16.1.tar.bz2 || { exit 1; }
 fi

 ## Unpack the source code.
 rm -Rf binutils-2.16.1 && tar xfvj binutils-2.16.1.tar.bz2 || { exit 1; }

 ## Enter the source directory and patch the source code.
 cd binutils-2.16.1 && cat ../../patches/binutils-2.16.1-PSP.patch | patch -p1 || { exit 1; }

 ## Create and enter the build directory.
 mkdir build-psp && cd build-psp || { exit 1; }

 ## Configure the build.
 ../configure --prefix="$PSPDEV" --target="psp" --enable-install-libbfd || { exit 1; }

 ## Compile and install.
 make clean && make -j 2 && make install && make clean || { exit 1; }

If nobody objects, my plan in the near future is to create a 1.0 branch in subversion for the old script, and start clean with this script.

Questions? Comments?
Back to top
View user's profile Send private message Visit poster's website
hlide



Joined: 10 Sep 2006
Posts: 750

PostPosted: Thu May 17, 2007 8:52 pm    Post subject: Reply with quote

that's a good way to modulate this script. I think there would be no objection.
Back to top
View user's profile Send private message
hlide



Joined: 10 Sep 2006
Posts: 750

PostPosted: Thu May 17, 2007 9:25 pm    Post subject: Reply with quote

well i failed with ncurse because I havn't /usr/lib/libncurses.so which is normal because i'm under cygwin and not under linux.

I think you toolchain need to be aware whether the host is windows or linux (or MacOX ?).
Back to top
View user's profile Send private message
hlide



Joined: 10 Sep 2006
Posts: 750

PostPosted: Thu May 17, 2007 9:53 pm    Post subject: Reply with quote

oh well, if a novice wants to install it, he/she would run into troubles :

Error : PSPDEV not set <== why not to propose one by default when not set ? usually /usr/local/pspdev as it is with the current psptoolchain.

I don't expect him/her to do a "export PSPDEV=/usr/local/pspdev" every time before running the toolchain.

I think you must fix those things before replacing the current psptoolchain with this one.

1) be sure it works like a charm under every (cygwin, linux, MacOS X)
2) don't let your scripts to be too petty. Whenever you have a variable not set, try to propose one by default so that people wanting just a working compiler to be able to compile a source they retrieve can do it without any knowledge about scripts
Back to top
View user's profile Send private message
ooPo
Site Admin


Joined: 17 Jan 2004
Posts: 2032
Location: Canada

PostPosted: Fri May 18, 2007 5:07 am    Post subject: Reply with quote

Quote:
well i failed with ncurse because I havn't /usr/lib/libncurses.so which is normal because i'm under cygwin and not under linux.

What does cygwin use for a termcap library? You need one in order to compile gdb. Would searching for libtermcap.so work instead?

Quote:
I don't expect him/her to do a "export PSPDEV=/usr/local/pspdev" every time before running the toolchain.

I expect a user to read the documentation and do what's needed to build and use a toolchain:

Code:
 ==================
  How do I use it?
 ==================

 1) Set up your environment by installing the following software:

  autoconf, automake, bison, flex, gcc, make, ncurses, patch, subversion, terminfo, wget

 2) Add the following to your login script:

  export PSPDEV=/usr/local/pspdev
  export PATH=$PATH:$PSPDEV/bin

 3) Run the toolchain script:

  ./toolchain.sh

I do agree that the error messages could be more helpful, but having the script use defaults and try to assume things will have the user end up with a toolchain that still won't work properly until they do the setup they should have done in the first place.
Back to top
View user's profile Send private message Visit poster's website
J.F.



Joined: 22 Feb 2004
Posts: 2906

PostPosted: Fri May 18, 2007 6:38 am    Post subject: Reply with quote

Go for it! Looks good to me.
Back to top
View user's profile Send private message AIM Address
hlide



Joined: 10 Sep 2006
Posts: 750

PostPosted: Fri May 18, 2007 8:11 pm    Post subject: Reply with quote

ooPo wrote:
Quote:
well i failed with ncurse because I havn't /usr/lib/libncurses.so which is normal because i'm under cygwin and not under linux.

What does cygwin use for a termcap library? You need one in order to compile gdb. Would searching for libtermcap.so work instead?


i do have ncurse, but libncurses.so is linux-ish ! there is no .so in cygwin (but rather .dll). I suppose you may test libncurses.dll.a which is in the same directory instead.

ooPo wrote:
Quote:
I don't expect him/her to do a "export PSPDEV=/usr/local/pspdev" every time before running the toolchain.

I expect a user to read the documentation and do what's needed to build and use a toolchain:

Code:
 ==================
  How do I use it?
 ==================

 1) Set up your environment by installing the following software:

  autoconf, automake, bison, flex, gcc, make, ncurses, patch, subversion, terminfo, wget

 2) Add the following to your login script:

  export PSPDEV=/usr/local/pspdev
  export PATH=$PATH:$PSPDEV/bin

 3) Run the toolchain script:

  ./toolchain.sh

I do agree that the error messages could be more helpful, but having the script use defaults and try to assume things will have the user end up with a toolchain that still won't work properly until they do the setup they should have done in the first place.


oh well i didn't read your readme.txt because i was expecting to use this psptoolchain the same way i did with the current one. So, yes, I should check if my login script has the right exports.
Back to top
View user's profile Send private message
hlide



Joined: 10 Sep 2006
Posts: 750

PostPosted: Fri May 18, 2007 8:13 pm    Post subject: Reply with quote

anyway, by just ignoring the ncurse check, i was able to run psptoolchaintool up to the end, so I suppose configure is able to retrieve ncurse even under cygwin.
Back to top
View user's profile Send private message
ooPo
Site Admin


Joined: 17 Jan 2004
Posts: 2032
Location: Canada

PostPosted: Fri May 18, 2007 8:25 pm    Post subject: Reply with quote

hlide wrote:
i do have ncurse, but libncurses.so is linux-ish ! there is no .so in cygwin (but rather .dll). I suppose you may test libncurses.dll.a which is in the same directory instead.

You're absolutely right. Duh. :) That's an easy enough change to make.
Back to top
View user's profile Send private message Visit poster's website
ooPo
Site Admin


Joined: 17 Jan 2004
Posts: 2032
Location: Canada

PostPosted: Fri May 18, 2007 8:37 pm    Post subject: Reply with quote

http://dev.oopo.net/files/psptoolchain-20070518.tar.bz2

Code:
#!/bin/sh
# check-ncurses.sh by Dan Peori (danpeori@oopo.net)

 ## Check for a ncurses library.
 ls /usr/lib/libncurses.* 1> /dev/null || { echo "ERROR: Install ncurses before continuing."; exit 1; }

This should work for everyone now, correct?
Back to top
View user's profile Send private message Visit poster's website
hlide



Joined: 10 Sep 2006
Posts: 750

PostPosted: Fri May 18, 2007 9:22 pm    Post subject: Reply with quote

i tried psp-gdb, psp-gdbtui and psp-insight which seem to work perfectly (psp-gdbtui should use ncurse, right ?)

so i think you can go now
Back to top
View user's profile Send private message
futaris



Joined: 28 Dec 2005
Posts: 45

PostPosted: Wed May 23, 2007 10:03 am    Post subject: Reply with quote

Don't suppose the script can be modified to download the .tar.gz and .tar.bz2's to a PSP_DL environment variable. Openembedded, buildroot, etc do something similar, so we can just download everything to one directory.
Back to top
View user's profile Send private message
weltall



Joined: 20 Feb 2004
Posts: 310

PostPosted: Wed May 23, 2007 3:31 pm    Post subject: Reply with quote

sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)
Back to top
View user's profile Send private message Visit poster's website
J.F.



Joined: 22 Feb 2004
Posts: 2906

PostPosted: Wed May 23, 2007 3:42 pm    Post subject: Reply with quote

weltall wrote:
sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)


Actually, the current stable gcc is now 4.2.0. :D
Back to top
View user's profile Send private message AIM Address
weltall



Joined: 20 Feb 2004
Posts: 310

PostPosted: Wed May 23, 2007 9:26 pm    Post subject: Reply with quote

J.F. wrote:
weltall wrote:
sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)


Actually, the current stable gcc is now 4.2.0. :D

i mean regarding available gcc patches for psp :P
Back to top
View user's profile Send private message Visit poster's website
hlide



Joined: 10 Sep 2006
Posts: 750

PostPosted: Thu May 24, 2007 4:10 am    Post subject: Reply with quote

weltall wrote:
J.F. wrote:
weltall wrote:
sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)


Actually, the current stable gcc is now 4.2.0. :D

i mean regarding available gcc patches for psp :P


those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.
Back to top
View user's profile Send private message
ooPo
Site Admin


Joined: 17 Jan 2004
Posts: 2032
Location: Canada

PostPosted: Thu May 24, 2007 4:15 am    Post subject: Reply with quote

weltall wrote:
sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)

You can still grab the old toolchain and patches:

svn export svn://svn.pspdev.org/psp/tags/psptoolchain-1.0.0

Are there problems with 4.1.0? When I asked nobody seemed to know of any outstanding issues.
Back to top
View user's profile Send private message Visit poster's website
Luke



Joined: 21 Sep 2006
Posts: 5

PostPosted: Thu May 24, 2007 4:58 am    Post subject: Reply with quote

Is this only an update to the toolchain-install-script or does this update the toolchain? If so, how do I do that? Can I simply run the new script?
Back to top
View user's profile Send private message
weltall



Joined: 20 Feb 2004
Posts: 310

PostPosted: Thu May 24, 2007 5:25 am    Post subject: Reply with quote

hlide wrote:
weltall wrote:
J.F. wrote:
weltall wrote:
sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)


Actually, the current stable gcc is now 4.2.0. :D

i mean regarding available gcc patches for psp :P


those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.

do you mean that those problems i experienced with it were made by not defining that command line argument?
Back to top
View user's profile Send private message Visit poster's website
ooPo
Site Admin


Joined: 17 Jan 2004
Posts: 2032
Location: Canada

PostPosted: Thu May 24, 2007 5:31 am    Post subject: Reply with quote

Luke wrote:
Is this only an update to the toolchain-install-script or does this update the toolchain? If so, how do I do that? Can I simply run the new script?

This is an update to the install script. If you already have a toolchain installed you don't need this - you're already up to date.

This new script just makes it easier to maintain updates in the future.
Back to top
View user's profile Send private message Visit poster's website
Luke



Joined: 21 Sep 2006
Posts: 5

PostPosted: Thu May 24, 2007 5:57 am    Post subject: Reply with quote

Cool, thanks! And by the way: Great work! Really. PSP-Programming rocks! Thanks for making this possible for me!
Back to top
View user's profile Send private message
hlide



Joined: 10 Sep 2006
Posts: 750

PostPosted: Thu May 24, 2007 7:03 am    Post subject: Reply with quote

weltall wrote:
hlide wrote:


those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.

do you mean that those problems i experienced with it were made by not defining that command line argument?


there is a topic somewhere covering the fact you can have wrong code because of strict aliasing and can be worse with -O3. Try to find it. This topic explains why and how to avoid those wrong code.

So yes I think your problems can be from the fact that you don't use this command line.
Back to top
View user's profile Send private message
weltall



Joined: 20 Feb 2004
Posts: 310

PostPosted: Fri May 25, 2007 3:41 am    Post subject: Reply with quote

hlide wrote:
weltall wrote:
hlide wrote:


those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.

do you mean that those problems i experienced with it were made by not defining that command line argument?


there is a topic somewhere covering the fact you can have wrong code because of strict aliasing and can be worse with -O3. Try to find it. This topic explains why and how to avoid those wrong code.

So yes I think your problems can be from the fact that you don't use this command line.


ok thanks when i have some time i will build the new toolchain and try again :)
Back to top
View user's profile Send private message Visit poster's website
Alek



Joined: 09 Mar 2007
Posts: 10

PostPosted: Fri May 25, 2007 6:37 am    Post subject: Reply with quote

Err... didn't say anything xD

Installing well, good job :)
Back to top
View user's profile Send private message
lmame



Joined: 27 May 2007
Posts: 2
Location: July City

PostPosted: Sun May 27, 2007 1:11 pm    Post subject: Reply with quote

Hi ^_^

First of all thank you for the toolchain and this new script, I'm quite new to psp dev world and for now I was just using oslib / pspsdk / cygwin under windows and when I saw your new script I decided to give it a try under Linux :)

I managed really simply to install and run things unit the last script until the last script is called: 008-insight-6.4.sh :
When the make -j 2 is run, I got this error (one line lost in an ocean of text):
Code:
/bf/toolchain/binutils-2.17/tk/unix/../generic/tk.h:96:23: X11/Xlib.h: No such file or directory

So I had to install X11 and run it again ;)

Perhaps it would be call to check also if X11 exists in one of your script, I guess it would be cool for noobs like me ;)

Anyway, now it's working :)
_________________
Life is full of love and peace ^_^
Back to top
View user's profile Send private message Visit poster's website
hlide



Joined: 10 Sep 2006
Posts: 750

PostPosted: Mon May 28, 2007 7:21 pm    Post subject: Reply with quote

ooPo :

gcc 4.2 is out, it has less regressions than gcc 4.1. It might be interesting to use it instead, provided that the current patches are still applyable.

gcc 4.2 : [ all : 252, serious : 129 ]
gcc 4.1 : [ all : 297, serious : 173 ]

EDIT : silly of me ! I wrote 2.x instead of 4.x !!!


Last edited by hlide on Mon May 28, 2007 8:43 pm; edited 1 time in total
Back to top
View user's profile Send private message
ufoz



Joined: 10 Nov 2005
Posts: 86
Location: Tokyo

PostPosted: Mon May 28, 2007 7:45 pm    Post subject: Reply with quote

Surely you mean 4.2?
Back to top
View user's profile Send private message Visit poster's website AIM Address MSN Messenger
hlide



Joined: 10 Sep 2006
Posts: 750

PostPosted: Mon May 28, 2007 8:41 pm    Post subject: Reply with quote

arfff yes it is 4.x not 2.x 8x !!!!
Back to top
View user's profile Send private message
hlide



Joined: 10 Sep 2006
Posts: 750

PostPosted: Thu Aug 09, 2007 8:31 pm    Post subject: Reply with quote

GRRRRRRRRRRRRRRRRR !!!!

the option "-mpreferred-stack-boundary=n" is not longer valid in this version !!!! still I modified some gcc files in gcc/config/mips directory so we can change the alignment requirement for stack !? does it means you use a different mips patch !?

now some of my sources are broken because of that.
Back to top
View user's profile Send private message
jimparis



Joined: 10 Jun 2005
Posts: 1179
Location: Boston

PostPosted: Fri Aug 10, 2007 2:44 am    Post subject: Reply with quote

Isn't relying on stack alignment a bit broken anyway? I thought even "-mpreferred-stack-boundary" is just an attempt not a guarantee. You could use memalign if alignment matters.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.ps2dev.org Forum Index -> PSP Development All times are GMT + 10 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group