Show enters and exits. Hide enters and exits.
| 04:55:03 | evan | gotta love living in an active earthquake zone. |
| 04:55:16 | evan | on the upside, the new loft survived the 5.0 earthquake test! |
| 05:03:37 | scoopr | :D |
| 05:04:25 | slava | hi evan |
| 05:10:18 | slava | evan: so I've started an overhaul of my code generator so that it keeps values in registers between basic blocks |
| 05:10:28 | slava | evan: should give a big speed boost for low-level loops and such |
| 13:49:47 | scoopr | just wow.. http://chris-lamb.co.uk/2008/01/24/can-you-get-cp-to-give-a-progress-bar-like-wget/ |
| 13:53:21 | rue | scoopr: Get the shotguns, I will find book a flight |
| 13:53:49 | scoopr | =) |
| 15:35:18 | dbussink | http://timetobleed.com/fixing-threads-in-ruby-18-a-2-10x-performance-boost/ |
| 15:51:57 | zinf | wonders what the current state of the art is for rubinius |
| 18:56:47 | evan | good morning |
| 19:00:29 | brixen | morning |
| 19:00:45 | dgtized | morning |
| 19:00:50 | brixen | any more rumblings down there? |
| 19:01:08 | brixen | and I don't mean did you have breakfast :) |
| 19:01:18 | evan | hah |
| 19:01:20 | evan | no |
| 19:01:24 | evan | there was a 3.1 aftershock |
| 19:01:31 | evan | but I didn't feel it |
| 19:01:45 | brixen | what was the 5.0 like? |
| 19:02:10 | evan | it's hard to explain what it's like when the whole room begins to shake independent of you |
| 19:02:25 | brixen | were you at the loft when it happened? |
| 19:02:27 | evan | because we're so used to things like rooms being stationary |
| 19:02:29 | evan | yep |
| 19:02:34 | evan | sitting on the couch, watching TV |
| 19:02:44 | brixen | so everything is ok with the building? |
| 19:02:49 | evan | yep! |
| 19:02:52 | brixen | cool! |
| 19:02:56 | evan | it survived it's first 5.0! |
| 19:02:57 | tarcieri | heh crazy |
| 19:03:09 | evan | nothing fell down |
| 19:03:12 | evan | which is the usual worry |
| 19:03:15 | evan | book cases, etc. |
| 19:03:40 | evan | so yeah, i'm trying to compare it to something... |
| 19:03:45 | evan | but i've got no frame of reference |
| 19:03:50 | evan | other than previous earthquakes |
| 19:04:09 | brixen | well good news the building weathered it well |
| 19:04:12 | evan | you hear the building creek as it moves |
| 19:04:51 | tarcieri | being in an earthquake sounds pretty absurd |
| 19:05:12 | evan | it is |
| 19:05:19 | evan | you know how humans are used to the ground being well, the ground. |
| 19:05:37 | evan | this flys in the face of millions of years of thought |
| 19:05:56 | tarcieri | heh |
| 19:06:01 | evan | sorta like being on a roller coaster, you have to be in one to know what it's like |
| 19:06:14 | tarcieri | earth moves, humans stay still instead of vice versa? |
| 19:06:24 | evan | yeah |
| 19:06:37 | evan | all of a sudden, things are moving underneith you |
| 19:06:39 | evan | and beside you |
| 19:06:48 | evan | because they all pick up the momentum before you do |
| 19:06:57 | evan | in a 10 second one, your body never picks up the momentum |
| 19:07:40 | evan | in LA though, whenever there is an earthquake, everyone things "uh oh, is this the BIG one?" |
| 19:07:54 | evan | thinks |
| 19:08:08 | evan | because we've overdue for a 7+ earthquake |
| 19:08:19 | evan | which, when it happens, will last up to 10 minutes |
| 19:09:33 | tarcieri | this is like the second time I've visited somewhere and a few weeks later there was a major earthquake |
| 19:10:03 | evan | you were just in LA? |
| 19:10:12 | tarcieri | err, just in San Fran I guess :/ |
| 19:10:31 | evan | well, thats close but no cigar |
| 19:10:40 | evan | SF is a different zone |
| 19:10:58 | tarcieri | yeah apparently I'm confused as to the location of the earthquake |
| 19:11:43 | evan | LA is built over the edge of the Pacific and North American plates |
| 19:11:50 | evan | like literally right on the edge |
| 19:11:58 | evan | about 30 miles of LA is moving north |
| 19:12:11 | tarcieri | yeah there's that little chunk of California that's gonna slide into the ocean some day |
| 19:12:13 | evan | if you've seen "Escape from LA", you know what I mean :D |
| 19:12:17 | tarcieri | heh |
| 19:14:50 | evan | so, i'm working on the background compiler thread |
| 19:15:03 | evan | which means i'm doing what will become the first stages of removing the GIL |
| 19:15:20 | evan | because there are some VM structures that need to be protected, so they can be used from both places |
| 19:15:26 | evan | at the same time |
| 19:15:30 | evan | like the SymbolTable |
| 19:32:43 | brixen | evan: ah cool |
| 19:37:28 | evan | i've added a SpinLock to symbol lookup |
| 19:37:54 | evan | i'm really glad I added all those nice C++ classes for dealing with locks |
| 19:55:48 | evan | hm, I wonder if I should track the module a CompiledMethod belongs to |
| 19:56:20 | brixen | what advantage is there? |
| 19:56:43 | brixen | I can't remember why we don't track it |
| 19:56:53 | evan | well, you can ask a CompiledMethod for it's full name then. |
| 19:56:59 | evan | i think i'm going to make that change |
| 19:57:03 | evan | but in a wider context |
| 19:57:03 | brixen | ah sure |
| 19:57:27 | evan | i've mentioned refactoring the Compiler to generate CodeBlock objects |
| 19:57:31 | evan | rather than CompiledMethod's |
| 19:57:48 | evan | since thats what they actually are |
| 19:58:04 | evan | ie, using a CompiledMethod for a block is, meh. |
| 19:58:11 | evan | it's a lot of needless info |
| 19:58:11 | brixen | yeah |
| 19:58:26 | evan | I think i'll look at this shortly. |
| 20:24:10 | brixen | I heard adium has twitter support too |
| 20:24:17 | brixen | how's the irc? |
| 21:03:07 | lypanov | um |
| 21:03:20 | lypanov | evan: adium fucking rocks |
| 21:05:08 | lypanov | wonders how rubinius is doing as jruby got all exciting again |
| 21:06:01 | evan | what ya doing in jruby? |
| 21:06:58 | evan | lypanov: things are good here |
| 21:07:05 | evan | working on the LLVM JIT again |
| 21:07:31 | evan | about to push the ability to run the JIT in the background |
| 21:08:12 | lypanov | yay! |
| 21:08:19 | lypanov | was so sad when i heard you wouldn't be using llvm |
| 21:08:56 | lypanov | evan: ahhh neat. so you can use a background low prio thread for the code gen and interpret if native isn't available? |
| 21:09:29 | evan | not sure what ya mean by interpret if native isn't availabel |
| 21:09:30 | evan | but yeah |
| 21:09:38 | evan | you can request a CompiledMethod to be JIT'd |
| 21:09:50 | evan | and a request is sent to a waiting thread |
| 21:11:36 | lypanov | if not already jitted i mean |
| 21:11:42 | lypanov | you do mixed mode already? |
| 21:11:45 | lypanov | or only jitted? |
| 21:11:46 | evan | yep |
| 21:11:48 | lypanov | cool |
| 21:11:48 | evan | mixed mode |
| 21:11:52 | evan | will always do mixed mode |
| 21:12:02 | evan | having the interpreter is just too valuable |
| 21:12:04 | evan | to fall back on |
| 21:12:26 | lypanov | impossible without i think |
| 21:12:34 | lypanov | you can't do anything interesting if you don't have a slow path |
| 21:12:59 | lypanov | evan: how are benches? |
| 21:13:06 | lypanov | evan: what sort of optz already in place? |
| 21:13:54 | evan | not many optz yet |
| 21:14:02 | evan | working mainly to get the JIT understanding all the bytecodes |
| 21:14:06 | evan | getting close to that now. |
| 21:14:19 | evan | one optz I did do is IC spying for primitives |
| 21:14:23 | evan | ie, type feedback |
| 21:14:24 | lypanov | did you end up doing the neat trick with llvm instrinsics for ops? |
| 21:14:44 | evan | so the JIT will read an IC, and if the method there is the Tuple::at primitive |
| 21:14:52 | evan | it inlines the code to perform Tuple::at at the call site |
| 21:15:04 | evan | so calling Tuple#at is nearly free |
| 21:15:04 | dgtized | evan: so is JIT auto enabled now, or what is the new flag? |
| 21:15:09 | evan | it's like accessing a C array |
| 21:15:15 | evan | dgtized: no, not auto enabled |
| 21:15:35 | evan | you have to pass some build flags |
| 21:15:46 | evan | and it's not automatically invoked yet by the VM either |
| 21:15:49 | evan | you have to do |
| 21:15:53 | evan | Rubinius.jit method(:blah |
| 21:15:54 | evan | ) |
| 21:16:02 | evan | or, coming shortly |
| 21:16:09 | evan | Rubinius.jit_soon method(:blah) |
| 21:16:31 | dgtized | ok, but the automatic decision to jit a method after it runs > 50 times is not currently enabled? |
| 21:17:29 | evan | correct. |
| 21:17:41 | evan | that will be added maybe today |
| 21:17:43 | evan | we'll see |
| 21:17:53 | boyscout | Only use the JIT calling debug trampoline in debug mode - 9de05c7 - Evan Phoenix |
| 21:17:53 | boyscout | Add background compiler thread to speed up performing JIT - 7469cc5 - Evan Phoenix |
| 21:18:12 | lypanov | evan: how is speed? |
| 21:18:20 | boyscout | CI: Build 7469cc5 failed. http://ci.rubini.us/rubinius/builds/7469cc54faa2169257a836744998577f17896d50 |
| 21:18:20 | evan | so far, good. |
| 21:18:24 | evan | bah! |
| 21:18:35 | evan | le sigh. |
| 21:18:40 | lypanov | la |
| 21:18:46 | lypanov | ( :P ) |
| 21:19:05 | boyscout | Remove extra qualifier - 51b8034 - Evan Phoenix |
| 21:19:22 | evan | at the beginning of the work, the Array#each benchmark took 8s with 5k iterations |
| 21:19:29 | evan | it currently takes 3.5s |
| 21:19:36 | evan | MRI takes 4.9s |
| 21:19:40 | lypanov | and interp? |
| 21:19:41 | evan | and it implements Array#each entirely in C |
| 21:19:50 | evan | ours is entirely in ruby |
| 21:20:08 | lypanov | where is the cost? |
| 21:20:17 | evan | which cost? |
| 21:20:19 | lypanov | as in why so slow? |
| 21:20:31 | lypanov | how is yarv? |
| 21:20:32 | evan | ya think 3.5s is still slow? |
| 21:20:54 | lypanov | i think it would probably be much faster in java, yes |
| 21:21:17 | evan | well, it calls out to a closure |
| 21:21:23 | evan | which is not inlined or viewed into at all. |
| 21:21:24 | lypanov | even mri? |
| 21:21:27 | evan | of course |
| 21:21:29 | lypanov | k. yeah. that'd be why |
| 21:21:39 | evan | as does yarv and jruby |
| 21:21:44 | evan | no one does block inlining |
| 21:21:46 | evan | i'm aware of. |
| 21:21:50 | lypanov | yeah. they all suck |
| 21:21:57 | evan | if you can do it |
| 21:22:00 | headius | some blocks will inline to some level in jruby |
| 21:22:01 | evan | please add it to rubinius :D |
| 21:22:12 | evan | headius: oh? like? |
| 21:22:17 | lypanov | headius: how does invoke fit into this? |
| 21:22:36 | lypanov | is the dispatch semantic enough for it to opt? |
| 21:22:44 | headius | blocks can inline into java-based #each impls, for example, if it's not polymorphic |
| 21:22:45 | lypanov | hopes that makes ANY sense |
| 21:23:15 | headius | and they could easily inline into the yield point if I did a little more work on it |
| 21:23:27 | headius | parts of them do already, but not all the way to the block body |
| 21:23:27 | evan | headius: well, thats kind of the wrong place to inline |
| 21:23:34 | evan | you don't want the block inlined into the #each method |
| 21:23:34 | lypanov | headius: so simple things like "sum" implementations for example? |
| 21:23:40 | evan | you want the #each method + block inlined into the caller. |
| 21:23:53 | headius | well, the problem is that the intervening method is going to be poly or megamorphic in almost every case |
| 21:23:59 | evan | yes |
| 21:24:09 | evan | block invoking is like 100% poly |
| 21:24:11 | headius | so unless you can inline across megamorphic calls, it's not a simple case |
| 21:24:23 | evan | well |
| 21:24:29 | evan | it's monomorphic if you inline into the caller |
| 21:24:43 | evan | but thats a much more complicated inline strategy |
| 21:24:45 | headius | but that means you need to inline and then re-profile and then inline again |
| 21:25:05 | lypanov | this is another time when i think i had the right idea and want to kick myself for not haing the motivation / pragmatism to pull it off |
| 21:25:06 | headius | hotspot doesn't even do that for static java code |
| 21:25:18 | dgtized | weird question, could you special case inline each by injecting the logic into the block, instead of trying to inline the each and the block into the caller? |
| 21:25:28 | dgtized | sorry the each logic into the block |
| 21:25:45 | evan | dgtized: how do you know what logic to put into the block? |
| 21:26:00 | evan | BUT i think you're onto something there |
| 21:26:08 | lypanov | dgtized: for some simple hardcoded cases eg homogenous array each yes |
| 21:26:19 | headius | dgtized: a better strategy may be to have a separate path in the code that assumes it's a specific kind of each, so the block can inline there |
| 21:26:23 | evan | the JIT could see that Array#each is being called |
| 21:26:23 | lypanov | but while cute, its kinda not really what is needed in long term |
| 21:26:40 | evan | and take the code for the block + the code for Array#each |
| 21:26:42 | evan | to create a new block |
| 21:26:51 | dgtized | right |
| 21:26:56 | evan | insert a guard |
| 21:27:02 | lypanov | (yay guards) |
| 21:27:02 | evan | then the new block is invoked once |
| 21:27:04 | evan | and looped within |
| 21:27:08 | evan | that actually is a great idea |
| 21:27:12 | headius | unless you're going to regenerate that for every each encountered by the block, I don't see how it's any better |
| 21:27:15 | evan | because that fixes the problem i've run into |
| 21:27:23 | evan | ie, variable scoping when inlining blocks |
| 21:27:36 | lypanov | headius: every each? |
| 21:27:41 | evan | headius: no no, other side of the coin |
| 21:27:55 | evan | you're generating methods at the call site for #each |
| 21:28:06 | evan | where, likely, #each is monomorphic |
| 21:28:10 | headius | that's what I was talking about |
| 21:28:11 | lypanov | right |
| 21:28:11 | evan | and the block is definitely monomorphic |
| 21:28:21 | lypanov | why would you need to regen? |
| 21:28:26 | lypanov | and whats "every each"? |
| 21:28:36 | lypanov | there is a variant for every inline block yes |
| 21:28:40 | lypanov | but whats the issue with that? |
| 21:28:50 | headius | there's potentially many each impls |
| 21:28:56 | boyscout | CI: 51b8034 success. 2683 files, 10323 examples, 32868 expectations, 0 failures, 0 errors |
| 21:29:01 | lypanov | but not exponential so don't see the issue |
| 21:29:03 | headius | and the block needs to pass into them, and then they need to do their invocation |
| 21:29:13 | lypanov | of course block inlining adds code mem |
| 21:29:17 | evan | headius: you're saying that the call to #each might not invoke Array#each everytime? |
| 21:29:21 | evan | thats what the guard is for |
| 21:29:37 | lypanov | um |
| 21:29:40 | lypanov | wait a sec |
| 21:29:42 | headius | yeah, and then this is either only useful for Array#each or you have to generate new for all the other eaches too |
| 21:30:09 | dgtized | this is trace jitting special cased for each |
| 21:30:12 | lypanov | i think its an interesting hardcoded case |
| 21:30:24 | headius | sure, the hardcoded case was exactly what I was trying to describe |
| 21:30:28 | lypanov | dgtized: my vm design is trace jitting. i hope rubydium gets there some day |
| 21:30:36 | dgtized | I think you could do it for anything that takes a block though |
| 21:30:45 | dgtized | it's just each is the easy one to think about |
| 21:30:47 | lypanov | but i guess it more likely that rubinius will :P |
| 21:30:59 | lypanov | dgtized: fully agree |
| 21:31:14 | evan | you're still having to invoke a block |
| 21:31:15 | headius | well, inlining across multiple levels is back to the polymorphic case again |
| 21:31:16 | lypanov | "once you've written it twice, then refactor it / make it a method" |
| 21:31:17 | lypanov | so |
| 21:31:20 | evan | but you're only having to do it once |
| 21:31:21 | lypanov | do two hardcoded opts |
| 21:31:22 | evan | which is nice. |
| 21:31:24 | lypanov | then abstract it |
| 21:31:33 | headius | even with a tracing jit you need to be able to guard/bail |
| 21:31:42 | evan | ok smart guys |
| 21:31:46 | evan | if i'm to continue |
| 21:31:48 | lypanov | headius: of course but that doesn't mean its not worthwhile :) |
| 21:31:49 | evan | i need food first :D |
| 21:32:01 | headius | lypanov: well, maybe and maybe not :) |
| 21:32:16 | headius | some of us have to balance production support against spending weeks working on inlining each + block :) |
| 21:32:22 | headius | there's only so many resources available |
| 21:32:56 | headius | now invokedynamic has an alternative solution to this |
| 21:33:21 | dgtized | supports the idea of food for evan to continue |
| 21:34:19 | evan | for jruby this idea matters less |
| 21:34:20 | headius | invokedynamic may allow specifying that particular frames in the call path are just intermediaries |
| 21:34:26 | evan | because hotspot is already doing so much |
| 21:34:32 | evan | for me, it could potentially matter a lot. |
| 21:34:34 | headius | and so they'll inline wheever they're called and profile |
| 21:34:42 | headius | evan: sure, or you could just not use blocks in core :D |
| 21:34:58 | dgtized | headius: mostly we aren't using blocks in core actually |
| 21:35:03 | evan | yeah hm. no. :/ |
| 21:35:04 | lypanov | make ruby useless to anything that needs speed |
| 21:35:14 | lypanov | its the reason why i'm probably gonna end up using scala no matter what happens ;) |
| 21:35:22 | evan | ok lunch. |
| 21:36:06 | lypanov | headius: i like invoke dynamic. don't get me wrong |
| 21:36:22 | lypanov | headius: if you can get that working its a far better option than messing around doing it all yourself |
| 21:36:35 | lypanov | just talking about rubinius |
| 21:38:05 | headius | lypanov: you should look at wycats' scala_ext project |
| 21:38:16 | headius | it's a set of nice DSLish stuff to define JRuby extensions in Scala |
| 21:38:49 | lypanov | wonders sometimes if wycats is a collective |
| 21:40:30 | headius | it works pretty well so far...though since scala can't define static methods we need to add a better way to define module methods (which are normally static right now) |
| 21:41:15 | lypanov | how does "object" make it into bytecode? |
| 21:48:35 | headius | lypanov: "object"? |
| 21:49:25 | lypanov | headius: shortcut in scala for singleton classes. |
| 21:49:33 | headius | ahh |
| 21:49:39 | headius | it's friggin weird |
| 21:50:01 | lypanov | read the artima interview with odersky on orthogonality of the design |
| 21:50:07 | lypanov | very good interview |
| 21:50:42 | lypanov | (http://www.artima.com/scalazine) |
| 21:50:56 | headius | it essentially gets turned into a singleton object accessible through a static accessor and then methods are called on that |
| 21:51:03 | headius | there's no way to create a normal static method in scala |
| 21:51:46 | lypanov | surely you can still use it? pretend the static accessor is a kind of proxy? |
| 21:51:54 | lypanov | sounds like a good way to do the mapping to me |
| 21:52:44 | headius | lypanov: the problem is that it hides that stuff |
| 21:52:54 | headius | you can't actually go after the static methods without a bunch of reflection tricks |
| 21:53:08 | headius | you don't get to see all that stuff, basicaly |
| 21:53:15 | lypanov | but don't you control both sides? |
| 21:53:30 | headius | no |
| 21:53:33 | lypanov | as in you can just say staticMethodProxy().myMethod() and pretend its static? |
| 21:53:34 | lypanov | ahhhh okay |
| 21:53:38 | headius | the singleton side is generated magically for you by scala compiler |
| 21:53:52 | headius | and calls to the object are calls through that singleton behind the scenes |
| 21:53:54 | lypanov | no i mean. you control the scala. you control the ruby. sounds like enough to make a mapping |
| 21:53:58 | headius | there's *lots* of that kind of magic |
| 21:54:15 | headius | well scala generates weird mangled classes and method names for that backend stuff |
| 21:54:27 | headius | they would be diffcult to use, if it's possible at all |
| 21:54:45 | lypanov | once used mono's system reflection emit to write a baby monomorphic ruby interpreter back when the ruby mono bindings only allowed for static method calling |
| 21:54:47 | headius | the way I would propose to handle this in JRuby is that we need a nice scala API for defining method objects directly |
| 21:55:01 | headius | which is fine...it's just that the autowiring stuff we have now depends on class/module methods being static |
| 21:55:12 | lypanov | many things are changing in scala 2.8.0 |
| 21:55:21 | lypanov | to the way .class files are generated |
| 21:55:58 | headius | what, scala changing major things in a minor release? you're kidding! |
| 21:56:17 | headius | I was amazed that 2.7.3 to 2.8 meant they added things like keyword args in constructors |
| 21:56:19 | lypanov | yeah, cus java never did that lol |
| 21:56:21 | headius | I don't really get their cycle |
| 21:56:48 | lypanov | i am getting told off by my girlfriend for not being a productive lil boy. so. i have to go code |
| 21:56:49 | headius | the scala backend class stuff is really kinda gross, when you know what it's doing |
| 21:56:54 | lypanov | (i love my fiancee) |
| 21:57:02 | headius | seeya! |
| 21:57:15 | lypanov | cya :) |
| 21:57:21 | lypanov | i used java back in 1.1 days btw |
| 21:57:22 | lypanov | :) |
| 21:57:34 | lypanov | so... yeah. minor release ;) |
| 21:58:01 | lypanov | used ruby in the 1.6.x days, which was probably the stupidest thing ever to do, he's still scarred |
| 21:58:06 | lypanov | gone :) |
| 22:22:12 | evan | yay! |
| 22:22:17 | evan | my LLVM patch was accepted |
| 22:22:21 | evan | no need to apply it manually anymore |
| 22:22:27 | evan | if you use SVN HEAD of LLVM |
| 22:30:55 | brixen | sweet |
| 22:40:14 | dgtized | do we have a benchmark that hits Regexp.escape nicely? |
| 22:40:32 | evan | brixen: so, you can now do |
| 22:40:38 | evan | Rubinius.jit_soon meth |
| 22:40:42 | dgtized | oh nm, I see we have one with a fixed string |
| 22:40:48 | evan | and it will be JIT'd in the background |
| 22:41:21 | brixen | evan: awesome |
| 22:47:02 | evan | ok, it's new chair day |
| 22:47:15 | evan | staples has some decent mesh chairs in stock up the road |
| 22:58:48 | boyscout | modified regexp benchmark to vary string length in addition to iterations - e2aa247 - Charles Comstock |
| 22:58:48 | boyscout | swap Regexp::ESCAPE_TABLE from Hash to Tuple for >2x speedup - 9c2dae4 - Charles Comstock |
| 23:01:06 | boyscout | CI: 9c2dae4 success. .................................................................................................... .................................................................................................... .................................................................................................... ............................................................................... |
| 23:01:12 | brixen | or not |
| 23:02:23 | brixen | interesting assertion failure |
| 23:02:28 | dgtized | I've been getting that same random LinkList failure locally at random intervals |
| 23:02:37 | dgtized | but I was getting before that patch so I don't think it's related |
| 23:02:50 | brixen | probably not |
| 23:02:57 | brixen | I've never seen it on os x |
| 23:03:22 | dgtized | it's happened at least twice here on an mspec run today, I don't think I saw it before this weekend |
| 23:03:34 | dgtized | but it wasn't reproduceable |
| 23:03:48 | dgtized | or particularly frequent |
| 23:05:37 | dgtized | whoops happened again, alright I will try and get it to happen under gdb |
| 23:06:33 | evan | if we can get the backtrace from it |
| 23:06:35 | evan | it should be easy |
| 23:06:42 | evan | looks like the LinkedList is getting corrupted |
| 23:10:02 | dgtized | wait how do we get gdb to run mspec these days? |
| 23:10:50 | dgtized | just --spec-debug or do those require -K -S options? |
| 23:10:55 | evan | --gdb |
| 23:10:56 | evan | i believe |
| 23:11:07 | evan | i have a special file I specify with -t |
| 23:11:16 | evan | that runs gdb --args bin/rbx "$@" |
| 23:13:49 | dgtized | trying --spec-gdb, --gdb is listed as option but appears to be broken |
| 23:14:04 | dgtized | hmm, and that hit the error but wasn't running gdb |
| 23:17:20 | brixen | --gdb was an option to work with shotgun |
| 23:17:25 | dgtized | hmm with -fs the "describe a running thread" specs seem to pause a lot, and the spec that triggers the failure is in conditionvariable |
| 23:19:54 | evan | ah |
| 23:19:57 | evan | use the -t method then. |
| 23:20:13 | evan | kendall :: git/rbx ยป cat bin/rbx-debug |
| 23:20:13 | evan | #!/bin/sh |
| 23:20:13 | evan | exec gdb --args bin/rbx "$@" |
| 23:20:20 | evan | chmod a+x bin/rbx-debug |
| 23:20:28 | evan | bin/mspec -t bin/rbx-debug ... |
| 23:21:49 | evan | oh der. |
| 23:21:51 | evan | i see it |
| 23:21:55 | evan | there is a bug in LinkedList |
| 23:22:33 | evan | oh hm. |
| 23:22:51 | dgtized | ok it's triggered in the conditionvariable specs, and I have a backtrace into it right now |
| 23:23:09 | evan | pastie it |
| 23:23:40 | dgtized | http://gist.github.com/113788 |
| 23:24:00 | evan | yep |
| 23:24:00 | evan | ok |
| 23:24:03 | evan | it's in the RootSet |
| 23:24:08 | evan | ok, one sec... |
| 23:25:18 | evan | oh der. |
| 23:25:22 | evan | it's the change cezarsa did yesterday. |
| 23:25:33 | evan | the fix is easy. |
| 23:26:31 | boyscout | initialize the thread with the GIL held - 54c12c6 - Evan Phoenix |
| 23:26:41 | evan | dgtized: that should fix it. |
| 23:27:05 | evan | probably need to move the root set to be per Thread |
| 23:28:39 | dgtized | I wonder why it was only triggering on linux |
| 23:29:31 | dgtized | that seems to have fixed it though |
| 23:29:32 | boyscout | CI: 54c12c6 success. 2683 files, 10323 examples, 32868 expectations, 0 failures, 0 errors |
| 23:30:27 | evan | ah |
| 23:30:39 | evan | because of the vastly different thread scheduler |
| 23:30:53 | evan | so the timing is completely different on linux |
| 23:31:18 | dgtized | which means rubinius threading errors are going to be more system dependent then? |
| 23:31:35 | evan | all threading errors are system dependent |
| 23:31:47 | evan | depends on what you call a rubinius threading error |
| 23:31:54 | dgtized | I'm not sure if they were with green threading is all I mean |
| 23:32:19 | headius | diggity |
| 23:32:19 | evan | if who were? |
| 23:32:22 | evan | i'm not following. |
| 23:33:13 | dgtized | I'm just curious how many race conditions will be exposed as people try their threaded code on platforms other then MRI's green threading |
| 23:34:26 | dgtized | likewise, since our scheduling is less consistant across systems, it's more likely to expose race conditions |
| 23:35:21 | dgtized | it's a good thing, but it's going to be unexpected for some folks |
| 23:36:52 | evan | dgtized: none that will cause VM crashes |
| 23:37:12 | evan | if people are dumb out their code and their code depends on the behavior of MRIs threading model |
| 23:37:17 | evan | then it could behave differently |
| 23:37:25 | evan | thankfull, jruby has sussed a lot of that out already |
| 23:38:22 | evan | ok |
| 23:38:22 | evan | bbiab. |
| 23:38:49 | headius | I don't mean this to sound discouraging, but mapping all the green threading nonsense to parallel threads is frigging hard |
| 23:39:01 | headius | even on top of making a VM that can do parallel threading |
| 23:39:13 | headius | hotspot does 95% of the hard stuff for us and it's still really hard |
| 23:40:52 | evan | headius: you mean the uniquely Green thread features |
| 23:40:53 | evan | yes? |
| 23:40:58 | evan | Thread#stop, etc. |
| 23:41:13 | evan | or do you mean timing issues |
| 23:41:22 | headius | well both really, since they're interrelated |
| 23:41:39 | headius | so the bad thread operations are one half, but then just memory model concerns are even trickier |
| 23:42:06 | headius | we have various guarantees from the JVM about when threads will see each others updates, etc, but it means we have to cause certain reads/writes to flush caches |
| 23:42:30 | headius | getting that end right in the VM is obviously really hard, but doing it right in the impl on top of that VM is almost as tricky |
| 23:43:04 | evan | ah |
| 23:43:04 | evan | i see |
| 23:43:13 | evan | you mean like the idea that String has no lock |
| 23:43:28 | evan | in MRI, that doesn't matter, since String's operations are atomic by threading nature |
| 23:43:36 | headius | well that we just punt on...it's not thread-safe and shouldn't be expected to be |
| 23:43:50 | headius | no sane parallel-threaded impl would force you to use mutexed collections for everything |
| 23:43:58 | headius | but the sneakier ones are things like call sites |
| 23:44:14 | headius | do you know that when thread B invalidates a method or a call site when thread A will see that? |
| 23:44:26 | headius | if it's in cache it may not get flushed...ever...for thread A |
| 23:44:42 | evan | ah. |
| 23:44:43 | evan | gotcha |
| 23:44:47 | headius | thread A would continue blissfully unaware that another thread on another core has invalidated something |
| 23:44:50 | headius | that stuff is all over the place |
| 23:45:04 | headius | we still struggle with it |
| 23:45:19 | headius | right now we pay the cost of making a lot of fields volatile and forcing cache flushes on almost all calls |
| 23:45:37 | headius | we're trying to find ways to coarsen that by defining what concurrency you should actually be able to expect from a parallel ruby |
| 23:45:53 | headius | in essence, we need to define what ruby's memory model is supposed to be in the presents of parallel execution |
| 23:46:13 | headius | presence |
| 23:46:19 | evan | sure |
| 23:46:31 | evan | those are definitely issues |
| 23:47:09 | evan | well, new chair time! |
| 23:47:16 | evan | so i can sit and work on these issues! |
| 23:47:21 | headius | john rose told us that our first doubling of performance might come from inlining, but a second double might come from addressing that "too much volatility" issue |