| View previous topic :: View next topic |
| Author |
Message |
sauron_le_noir
Joined: 05 Jul 2008
Posts: 229
|
 Posted: Tue Aug 26, 2008 1:16 am Post subject: Malloc and Realloc on psp |
 |
|
Is there a function available that give you the remaining space available for a malloc or realloc function ? the new sumatra pdf load a load of resource in main memory and
for some pdfs (pdf with large image) the realloc function get a out of mem |
|
| Back to top |
|
 |
moonlight
Joined: 26 Oct 2005
Posts: 567
|
| Posted: Tue Aug 26, 2008 2:20 am Post subject: |
|
|
| Try mallinfo, maybe it works. |
|
| Back to top |
|
|
cory1492
Joined: 10 Dec 2004
Posts: 216
|
| Posted: Tue Aug 26, 2008 12:49 pm Post subject: |
|
|
| I think the malloc/free or realloc still has some leakage, if you are getting tight on memory you may want to keep that in mind. |
|
| Back to top |
|
|
hlide
Joined: 10 Sep 2006
Posts: 750
|
| Posted: Tue Aug 26, 2008 3:51 pm Post subject: |
|
|
| you mean fragmentation ? because fragmentation and leakage aren't the same thing. A leakage would be unacceptable and I cannot imagine libc to have a defective malloc and realloc. On the other hand, malloc and realloc might use an algorithm which can lead to a lot of fragmented free blocks of small size under some conditions, preventing you from allocating a big one even if total free space is enough. |
|
| Back to top |
|
|
sauron_le_noir
Joined: 05 Jul 2008
Posts: 229
|
| Posted: Wed Aug 27, 2008 3:45 am Post subject: |
|
|
it's the second one there are many little block available but not one big enought
that is the function creating this.it increment by 4k until it find a buffer large enought
to read the PDF stream resource.
Pdf containing a lot of streams ( essentialy pdfs with a lot a images) got a out of memory error. AM i right the psp doesn't have virtual memory or DAT (dynamic address translator) ?
Here is the function causing the trouble
| Code: |
*
* Utility function to consume all the contents of an input stream into
* a freshly allocated buffer; realloced and trimmed to size.
*/
enum { CHUNKSIZE = 1024 * 4 };
fz_error *
fz_readall(fz_buffer **bufp, fz_stream *stm)
{
fz_error *error;
fz_buffer *real;
unsigned char *newbuf;
unsigned char *buf;
int len;
int pos;
int n;
len = 0;
pos = 0;
buf = nil;
while (1)
{
if (len - pos == 0)
{
if (len == 0)
len = CHUNKSIZE;
else
len *= 2;
newbuf = fz_realloc(buf, len);
if (!newbuf)
{
fz_free(buf);
return fz_throw("outofmem: scratch buffer");
}
buf = newbuf;
}
error = fz_read(&n, stm, buf + pos, len - pos);
if (error)
{
fz_free(buf);
return fz_rethrow(error, "cannot read data");
}
pos += n;
if (n < CHUNKSIZE)
{
if (pos > 0)
{
newbuf = fz_realloc(buf, pos);
if (!newbuf)
{
fz_free(buf);
return fz_throw("outofmem: scratch buffer");
}
}
else newbuf = buf;
real = fz_malloc(sizeof(fz_buffer));
if (!real)
{
fz_free(newbuf);
return fz_throw("outofmem: buffer struct");
}
real->refs = 1;
real->ownsdata = 1;
real->bp = newbuf;
real->rp = newbuf;
real->wp = newbuf + pos;
real->ep = newbuf + pos;
real->eof = 1;
*bufp = real;
return fz_okay;
}
}
} |
[/code]
Last edited by sauron_le_noir on Wed Aug 27, 2008 5:40 pm; edited 1 time in total |
|
| Back to top |
|
|
sauron_le_noir
Joined: 05 Jul 2008
Posts: 229
|
| Posted: Wed Aug 27, 2008 4:28 am Post subject: |
|
|
[Can somebody explain me or point me a tutorial that explain the organisation of the memory partition on a psp (can be technical i'm a old os/2 device driver develloper)
also a explanation of the meminfo whould be great
| Code: |
host0:/> meminfo
Memory Partitions:
N | BASE | SIZE | TOTALFREE | MAXFREE | ATTR |
---|------------|----------|-----------|-----------|------|
1 | 0x88000000 | 3145728 | 289792 | 268544 | 000C |
2 | 0x08800000 | 54525952 | 782336 | 524288 | 000F |
3 | 0x88000000 | 3145728 | 289792 | 268544 | 000C |
4 | 0x88300000 | 1048576 | 1048576 | 1048576 | 000C |
5 | 0x08400000 | 4194304 | 4194304 | 4194304 | 000F |
6 | 0x08800000 | 54525952 | 782336 | 524288 | 000F |
8 | 0x8BC00000 | 0 | 0 | 0 | 000C |
10 | 0x8BC00000 | 4194304 | 4194304 | 4194304 | 000C |
11 | 0x8BC00000 | 0 | 0 | 0 | 000C |
host0:/> warning: cannot realloc 286720 bytes
+ stream/stm_misc.c:91: fz_readall(): outofmem: scratch buffer
| mupdf/pdf_stream.c:510: pdf_loadstream(): cannot load stream into buffer (366)
| mupdf/pdf_resources.c:165: preloadxobject(): cannot load image resource 366
| mupdf/pdf_resources.c:430: pdf_loadresources(): cannot load xobject resource
| mupdf/pdf_page.c:233: pdf_loadpage(): cannot load page resources
|
i guess ATTR give me the type of memory kernel or is partition number 2 the user partition ?
in the meminfo i see that i have 286720 byte free but the realloc failed
Initializing triMemory
allocating 557056 bytes, in 2176 blocks
allocated at address 0x4000000
allocating 557056 bytes, in 2176 blocks
allocated at address 0x4088000
Initializing triInput
allocating 3072 bytes, in 12 blocks
allocated at address 0x4110000
allocating 5120 bytes, in 20 blocks
allocated at address 0x4110c00
allocating 475136 bytes, in 1856 blocks
allocated at address 0x4112000
why have i 0x400000 ,4112000 etc as adrs it is display with a printf (" %p" |
|
| Back to top |
|
|
sauron_le_noir
Joined: 05 Jul 2008
Posts: 229
|
|
| Back to top |
|
|
sauron_le_noir
Joined: 05 Jul 2008
Posts: 229
|
| Posted: Wed Aug 27, 2008 6:26 am Post subject: |
|
|
grrr malloc_hook are gone in 4.3.1
----> not use malloc hooks. They cannot work reliably and will disappear
in the next malloc implementation. Ignore them.
--
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
http://www.linuxjournal.com/article/6390
malloc_stats() works on psp |
|
| Back to top |
|
|
hlide
Joined: 10 Sep 2006
Posts: 750
|
| Posted: Wed Aug 27, 2008 3:49 pm Post subject: |
|
|
sauron_le_noir, again, please use tag | Code: | | [code]your code here[/code] |
to put your code therein. Any code you post here is unreadable because the indentation is lost. Nobody will read your code if your persist this way. |
|
| Back to top |
|
|
sauron_le_noir
Joined: 05 Jul 2008
Posts: 229
|
| Posted: Wed Aug 27, 2008 5:41 pm Post subject: |
|
|
| it's done |
|
| Back to top |
|
|
cory1492
Joined: 10 Dec 2004
Posts: 216
|
| Posted: Fri Aug 29, 2008 5:15 am Post subject: |
|
|
| hlide wrote: | | you mean fragmentation ? because fragmentation and leakage aren't the same thing. A leakage would be unacceptable and I cannot imagine libc to have a defective malloc and realloc. On the other hand, malloc and realloc might use an algorithm which can lead to a lot of fragmented free blocks of small size under some conditions, preventing you from allocating a big one even if total free space is enough. |
All I know for sure at this point is, if I malloc+realloc until all available memory is used (in small chunks), then free it, my next malloc cannot be the same amount I initially allocated. Whether this is an "on purpose" function of malloc I couldn't say, but it sure looks like some kind of leakage (or error) is occurring to me and is something I figured would be worth mentioning to sauron_le_noir.
Due to this behavior, I for one try to avoid malloc. |
|
| Back to top |
|
|
Raphael
Joined: 17 Jan 2006
Posts: 646
Location: Germany
|
| Posted: Fri Aug 29, 2008 7:03 am Post subject: |
|
|
| sauron_le_noir wrote: |
Initializing triMemory
allocating 557056 bytes, in 2176 blocks
allocated at address 0x4000000
allocating 557056 bytes, in 2176 blocks
allocated at address 0x4088000
Initializing triInput
allocating 3072 bytes, in 12 blocks
allocated at address 0x4110000
allocating 5120 bytes, in 20 blocks
allocated at address 0x4110c00
allocating 475136 bytes, in 1856 blocks
allocated at address 0x4112000
why have i 0x400000 ,4112000 etc as adrs it is display with a printf (" %p" |
Those allocs are the ones from triVAlloc for VRAM (which allocate in 0x4000000-0x4200000). For the normal triMalloc log see triMemoryLog.txt
_________________
<Don't push the river, it flows.>
http://wordpress.fx-world.org - my devblog
http://wiki.fx-world.org - VFPU documentation wiki
Alexander Berl |
|
| Back to top |
|
|
sauron_le_noir
Joined: 05 Jul 2008
Posts: 229
|
| Posted: Sat Aug 30, 2008 3:55 am Post subject: |
|
|
I've found why sumatra pdf fails on some pdfs it is not a question of numbers of pages i have a redbook from IBM with 631 rendering wihout any problems but i have a pdf with 200 pages every page is a bitmap ARGB 480x272 and it breaks. Sumatra pdf library store all the resource in a pdf store in memory so the last pdf get a out of mem when reading the page 134. I'm rewritting the pdf_store code of the library to accept
to use memory or Memory stick swap space for his pdf store.
So i hope to be able to render every pdf on the psp that sumatra can render on a pc.
I've also investigated alternative malloc,free,realoc than standard libgc
It is a work in progress...... keep in touch :p:p:p
And raphael your library is really great !!!! i have solved the bitmap problems H > 1024 and W > 1024
I have a ARGB with a bitmap greater than 1024x1024
i just created a bitmap of 480x272 and extract the portion form the big buffer so problem solve and it is very fast. |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
