This discussion is archived
6 Replies Latest reply: Nov 26, 2008 11:11 AM by 628326 RSS

what is mapLN and why is it eating all our space :)

628326 Newbie
Currently Being Moderated
here's a DbPringLog from a few months ago:

<pre>
[xoopit@xda-004 ###]$ java -jar ~/###/je-3.3.74.jar DbPrintLog -s 0x00000000 -e 0x0000000a -S
<DbPrintLog>
Log statistics:
type total provisional total min max avg entries
count count bytes bytes bytes bytes as % of log
LN_TX 219,704 0 26,803,878 117 132 121 44.7
MapLN 14 0 956 59 72 68 0
NameLN_TX 4 0 292 69 79 73 0
DupCountLN_TX 73,025 0 5,915,025 81 81 81 9.9
DupCountLN 211 211 11,816 56 56 56 0
FileSummaryLN 10 0 69,506 91 26,681 6,950 0.1
IN 2,033 76 8,213,996 67 6,368 4,040 13.7
BIN 4,064 3,957 12,297,614 57 6,319 3,025 20.5
DIN 799 54 253,737 197 687 317 0.4
DBIN 1,334 1,333 3,626,993 166 6,389 2,718 6
Root 4 0 326 62 88 81 0
Commit 73,238 0 2,783,044 38 38 38 4.6
CkptStart 2 0 81 39 42 40 0
CkptEnd 3 0 256 84 87 85 0
BINDelta 35 0 19,110 75 975 546 0
FileHeader 7 0 266 38 38 38 0
key/data 13,621,638 (22.7)

Total bytes in portion of log read: 59,996,896
Total number of entries: 374,487

Per checkpoint interval info:
lnTxn ln mapLNTxn mapLN end-end end-start start-end maxLNReplay ckptEnd
0 0 0 1 1,345 563 782 0 0x0/0x541
20,529 0 0 7 5,516,716 4,949,678 567,038 20,529 0x0/0x5432ed
128,339 0 0 6 34,716,133 54,481,939 0 128,339 0x7/0x392d2
70,836 0 0 0 19,765,806 19,765,806 0 70,836 0xa/0x0
</DbPrintLog>
</pre>

then a few days ago:

<pre>
[xoopit@xda-004 ###]$ java -jar ~/###/je-3.3.74.jar DbPrintLog -s 0x003ca27f -e 0x003cb06b -S
<DbPrintLog>
Log statistics:
type total provisional total min max avg entries
count count bytes bytes bytes bytes as % of log
LN_TX 534 0 51,086 86 108 95 0.1
LN 267,341 0 22,505,538 77 93 84 25.3
MapLN 9 0 668,956 74,226 74,539 74,328 0.8
DupCountLN_TX 178 0 11,393 63 65 64 0
DupCountLN 22 0 912 40 42 41 0
FileSummaryLN 7,669 0 693,427 72 2,462 90 0.8
IN 9,182 166 33,886,067 81 6,428 3,690 38.1
BIN 7,731 491 24,977,397 49 6,478 3,230 28.1
DIN 909 9 2,468,396 144 6,445 2,715 2.8
DBIN 619 56 3,555,037 136 6,494 5,743 4
Root 1 0 8,260 8,260 8,260 8,260 0
Commit 178 0 6,230 35 35 35 0
CkptStart 7 0 224 32 32 32 0
BINDelta 153 0 21,740 42 1,284 142 0
DupBINDelta 14 0 4,686 74 1,324 334 0
Trace 114 0 20,812 76 306 182 0
FileHeader 10 0 380 38 38 38 0
key/data 13,167,649 (14.8)

Total bytes in portion of log read: 88,880,541
Total number of entries: 294,671

Per checkpoint interval info:
lnTxn ln mapLNTxn mapLN end-end end-start start-end maxLNReplay ckptEnd
534 267,341 0 9 30,270,000,000 30,258,643,275 0 267,875 0x3cb06b/0x0
</DbPrintLog>
</pre>

and from today:

<pre>
xoopit@xda-004 ###]$ java -jar ~/###/je-3.3.74.jar DbPrintLog -s 0x004207c0 -e 0x004207c9 -S
<DbPrintLog>
Log statistics:
type total provisional total min max avg entries
count count bytes bytes bytes bytes as % of log
LN 44 0 3,507 78 93 79 0
MapLN 32 0 76,040,837 295,485 8,572,698 2,376,276 96.7
FileSummaryLN 156 0 9,362 30 220 60 0
IN 36 2 110,580 46 6,178 3,071 0.1
BIN 43 43 88,914 79 6,478 2,067 0.1
Root 11 0 2,405,444 218,657 218,708 218,676 3.1
CkptStart 1 0 31 31 31 31 0
CkptEnd 1 0 71 71 71 71 0
Trace 13 0 1,146 51 282 88 0
FileHeader 10 0 380 38 38 38 0
key/data 1,967 (0)

Total bytes in portion of log read: 78,660,272
Total number of entries: 347

Per checkpoint interval info:
lnTxn ln mapLNTxn mapLN end-end end-start start-end maxLNReplay ckptEnd
0 44 0 22150,110,853,335150,109,212,264 1,641,071 44 0x4207c6/0xd0557
0 0 0 10 29,146,665 29,146,665 0 5 0x4207c9/0x0
</DbPrintLog>
</pre>

as you can see 96.7% of our store is taken up by these mapLN entries. we haven't found any documentation as to what these things are. this is killing one of our BDB stores because in the last 24 hours we've generated around 10,000 new .jdb files when we used to have a few hundred...

we're using je-3.3.74

we upgraded to je-3.3.74 (from 3.3.62) on November 3rd but this particular issue didn't show up until around 24 hours ago.

thanks for taking a look ~j

Edited by: jules | xoopit on Nov 13, 2008 11:59 AM

Edited by: jules | xoopit on Nov 13, 2008 1:11 PM
  • 1. Re: what is mapLN and why is it eating all our space :)
    628326 Newbie
    Currently Being Moderated
    i should also mention that the je.properties for this bdb is pretty much stock

    <pre>
    je.maxMemoryPercent=5
    </pre>
  • 2. Re: what is mapLN and why is it eating all our space :)
    628326 Newbie
    Currently Being Moderated
    one more item of note for those of you familiar with our typical BDB questions. this is a much smaller BDB that we use to store very small amounts of data, not our "chunk store".
  • 3. Re: what is mapLN and why is it eating all our space :)
    greybird Expert
    Currently Being Moderated
    Hi Jules,

    MapLNs are per-Database metadata, and they can become large because they contain the log cleaner's housekeeping information. For some reason, it looks like your actual (utilized) data in this environment has become small (you deleted almost everything?) but the obsolete MapLNs have not been cleaned yet.

    Could you please run DbSpace as follows and send the output to my email address, mark.hayes at the obvious .com?

    java -jar je.x.y.z.jar DbSpace -h <dir> -d

    I am in and out right now, so I may not reply for a couple hours, but I will reply this afternoon or evening.

    Thanks,
    --mark                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
  • 4. Re: what is mapLN and why is it eating all our space :)
    628326 Newbie
    Currently Being Moderated
    thanks mark! email sent

    ~j
  • 5. Re: what is mapLN and why is it eating all our space :)
    greybird Expert
    Currently Being Moderated
    Hi All,

    We've been working on this with the Xoopit folks and I wanted to follow up and post the resolution, since it potentially impacts everyone using JE 3.3.x.

    There is a bug in JE 3.3.74 and earlier, in all versions of the 3.3.x product. The fix for this is in JE 3.3.75, which currently is available on request. We haven't decided when we'll update our download site with this updated release. If you would like the updated release, please send email to mark.hayes at the obvious .com (oracle).

    Here's the change log entry for the bug, which should explain what you need to know:

    -------------
    Fix a bug that caused the space taken by internal metadata in JE log files to increase over a long period of time. The rate of increase was slow in most cases, but in at least one observed case became rapid after a long time period and after the log cleaner became backlogged. To determine whether your JE log exhibits this problem, run

    java -jar je.x.y.z.jar DbPrintLog -h DIR -S

    and examine the line labeled MapLN on the left. If the amount of the log taken by MapLNs is 10% or greater, or if you see this number increasing steadily over time, then your application is probably experiencing this problem.

    By installing JE 3.3.75 or later, the excess disk space will automatically be reclaimed over time, as ordinary checkpoints and log cleaning occur. If you wish to recreate your database rather than wait for this to occur gradually, you can use DbDump and DbLoad to do so.

    We'd like to express our appreciation and sincere thanks to Jules and the other folks at Xoopit who reported this problem and helped us to diagnose it. We would not have found or fixed this problem as quickly as we did without their help.
    -------------

    For reference, the support ticket # for this problem is: #16610

    If you have further questions, please reply to this forum post.

    Thanks,
    --mark                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
  • 6. Re: what is mapLN and why is it eating all our space :)
    628326 Newbie
    Currently Being Moderated
    Forgot to followup that we're running 3.3.75 now and everything is doing great. Thanks sleepycat!

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points