Closed Bug 324005 Opened 19 years ago Closed 18 years ago

Make "new" not throw [OOM crash in canvas]

Categories

(Firefox :: General, defect)

1.5.0.x Branch
x86
Linux
defect
Not set
normal

Tracking

()

RESOLVED DUPLICATE of bug 353144

People

(Reporter: guninski, Unassigned)

Details

(Keywords: crash, Whiteboard: [sg:nse])

Attachments

(1 file)

there is strange crash which seems to blow the stack related to canvas.
1.5 and 1.5.0.1 are affected, trunk isn't (trunk seems to accept the canvas,
at least adds scrollbars).
probably an integer overflow.

<canvas id="canvas" width="32767" height="16383"></canvas>

may be related to Bug 322215

testcase to follow.



Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1220622656 (LWP 22117)]

#0  0x087ea893 in crmf_encoder_out ()
#1  0x087eaaca in crmf_encoder_out ()
#2  0x087eabf6 in crmf_encoder_out ()
#3  0x087d3cf3 in crmf_encoder_out ()
#4  0x087da1e8 in crmf_encoder_out ()
#5  0x087da026 in crmf_encoder_out ()
#6  0x087d2397 in crmf_encoder_out ()
#7  0x087d25a0 in crmf_encoder_out ()
#8  0x087d2661 in crmf_encoder_out ()
#9  0x087ce72c in crmf_encoder_out ()
#10 0x087ce6f2 in crmf_encoder_out ()
#11 0x08370bbe in nsReadingIterator<unsigned short>::advance ()
#12 0x08370051 in nsReadingIterator<unsigned short>::advance ()
#13 0x084b4468 in nsReadingIterator<unsigned short>::advance ()
#14 0x084b4437 in nsReadingIterator<unsigned short>::advance ()
#15 0x084b4273 in nsReadingIterator<unsigned short>::advance ()
#16 0xb7f4240b in XPTC_InvokeByIndex ()
Version: unspecified → 1.5 Branch
Attached file testcase (deleted) —
Flags: testcase+
The amount we're trying to allocate is valid (just large); operator new is throwing std::bad_alloc, and we don't do exception handling.. and things blow up.

Stack:

 kernel32.dll!7c81eb33() 	
 kernel32.dll!7c81eb33() 	
>msvcr71d.dll!_nh_malloc_dbg(unsigned int nSize=1, int nhFlag=3, int nBlockUse=1236500, const char * szFileName=0x02c26aa8, int nLine=47282736)  Line 267 + 0x7	C
 msvcr71d.dll!_CxxThrowException(void * pExceptionObject=0x0012de30, const _s__ThrowInfo * pThrowInfo=0x104f6150)  + 0x39	C++
 msvcp71d.dll!std::_Nomemory()  Line 10	C++
 gkgfxwin.dll!operator new(unsigned int size=1610514432)  Line 15	C++
 gkgfxwin.dll!operator new[](unsigned int count=1610514432)  Line 7 + 0x9	C++
 gkgfxwin.dll!nsImageWin::Init(int aWidth=32767, int aHeight=16383, int aDepth=24, nsMaskRequirements aMaskRequirements=nsMaskRequirements_kNeeds8Bit)  Line 184	C++
 gkgfxwin.dll!gfxImageFrame::Init(int aX=0, int aY=0, int aWidth=32767, int aHeight=16383, int aFormat=2, unsigned short aDepth=24)  Line 142	C++
 gklayout.dll!nsHTMLCanvasElement::UpdateImageContainer(int forceCreate=0)  Line 328 + 0x19	C++
 gklayout.dll!nsHTMLCanvasElement::GetCanvasImageContainer(imgIContainer * * aImageContainer=0x02d4b560)  Line 355 + 0xa	C++
 gklayout.dll!nsHTMLCanvasFrame::Reflow(nsPresContext * aPresContext=0x02d4cb00, nsHTMLReflowMetrics & aMetrics={...}, const nsHTMLReflowState & aReflowState={...}, unsigned int & aStatus=0)  Line 93 + 0x1c	C++
 gklayout.dll!nsLineLayout::ReflowFrame(nsIFrame * aFrame=0x5ffe8000, unsigned int & aReflowStatus=1236640, nsHTMLReflowMetrics * aMetrics=0x01a16a4e, int & aPushedFrame=)  Line 996	C++
...

Or rather, that's the case/stack on windows.  Haven't tried linux.
continue to crash with 1.5.0.1rc1 on linux - haven't tried winblows.

maybe linux specific?
/me believes something should be done about "new".

crashing in email is not very pleasant.

and coding style like this is not very wise:
./gfx/src/windows/nsImageWin.cpp

    // 32-bit align each row
    mARowBytes = (mARowBytes + 3) & ~0x3;
    mAlphaBits = new unsigned char[mARowBytes * aHeight];
    if (!mAlphaBits) {
      delete[] mBHead;
// ^^^^^^^ not reached at least with gcc
      mBHead = nsnull;
      delete[] mImageBits;

1.5.0.1rc1 doesn't crash on wine with the testcase.

I don't get a crash on OS X, using Fx 1.5, FWIW.
(In reply to comment #6)
> I don't get a crash on OS X, using Fx 1.5, FWIW.
> 

i crash on linux from 2 different boxes.
may be library specific, though doubt it.
I see a crash on linux with the trunk; same issue, operator new is throwing, and the runtime just calls abort() because we don't have exception handling support.  This should probably morph into a bug regarding fixing our usage of |operator new| vs. |PR_Malloc|, or get our own operator new impl that doesn't throw.  (Or start catching exceptions, if we could take the size and perf hit!)
vlad: do you crash on linux with 1.5.0.1rc1 ?

i don't crash on trunk, but firefox consumes about 2.1GB virtual memory (have a lot of swap).

no exception from |new|.

may be the amount ot virtual memory matters.

Group: security
Summary: probably an integer overflow in canvas (or reflow) → Make "new" not throw [OOM crash in canvas]
Whiteboard: [sg:nse]
am i the only lamer who can't tell abort() from a fuck?
mean that i believe that i am familiar with the side effects of an ***experienced*** c++ programer trying to do new char[ALITTLELESSTHANINFITY] without using nothrow and/or exceptions.
Keywords: crash
Bug 353144 has a lot more details now, so I mark this as a dupe of it.

*** This bug has been marked as a duplicate of 353144 ***
Status: NEW → RESOLVED
Closed: 18 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: