Show enters and exits. Hide enters and exits.
| 00:07:09 | postmodern | hey your tests for Socket#getnameinfo are wrong |
| 00:07:19 | postmodern | that returns the first name associated with localhost |
| 00:07:26 | postmodern | for those of us who name our computers |
| 00:44:25 | boyscout | Add timed locking and lock interruption - 34979d1 - Evan Phoenix (hydra) |
| 00:44:26 | boyscout | nil out @owner, then unlock - df2b000 - Evan Phoenix (hydra) |
| 00:44:26 | boyscout | Catch exceptions inside the background thread - f765501 - Evan Phoenix (hydra) |
| 00:44:26 | boyscout | Pass down the thread's state. Fixes thread alloc corruption bug. - d474224 - Evan Phoenix (hydra) |
| 00:44:26 | boyscout | Change call protocol to remove Dispatch&. Make IC thread-safe. - 45a2de9 - Evan Phoenix (hydra) |
| 05:20:49 | brixen | woot! thread safe ICs |
| 05:21:15 | brixen | opens xmas presents |
| 05:42:57 | evan | brixen: you make it to carlsbad? |
| 05:43:55 | brixen | escondido, but yes :) |
| 05:44:03 | brixen | got in about an hour ago |
| 05:44:15 | brixen | freeway land is so so crazy |
| 05:44:24 | evan | welcome to my hood! :) |
| 05:44:31 | brixen | I took 5 to 210 to (via some crazy shit) 15 |
| 05:44:36 | brixen | heh |
| 05:44:52 | brixen | and you're going to be in my hood this weekend, crazy :) |
| 05:45:08 | evan | hehe |
| 05:45:11 | evan | yep! |
| 05:45:20 | evan | you think i'll need to rent a car? |
| 05:45:29 | evan | i told ya where the wedding is, right? |
| 05:45:35 | brixen | ahh hm, yeah I think you probably will want to |
| 05:45:47 | evan | ok |
| 05:45:52 | brixen | yeah, there is pretty good bus access to that area |
| 05:46:02 | brixen | but I don't think you want to take the bus in wedding duds :) |
| 05:46:03 | evan | it's hot right? maybe i'll splurg for a Ferrari |
| 05:46:05 | evan | :D |
| 05:46:07 | brixen | hah |
| 05:46:11 | brixen | do it! :) |
| 05:46:36 | brixen | supposed to be in the 70's this weekend |
| 05:46:38 | tarcieri | hey uhh |
| 05:46:49 | brixen | tarcieri: uhh hey |
| 05:46:55 | evan | looks up how much it is |
| 05:46:55 | tarcieri | did you guys read Charlie's shit on Oracle v. Google? |
| 05:47:01 | brixen | not yet |
| 05:47:10 | brixen | I've been trying to relax |
| 05:47:11 | tarcieri | I'm curious if Rubinius is in violation of: Interpreting Functions Utilizing A Hybrid Of Virtual And Native Machine Instructions (6,910,205) |
| 05:47:14 | tarcieri | :( |
| 05:47:17 | tarcieri | fucking software patents |
| 05:47:25 | brixen | fuck scoracle |
| 05:48:13 | evan | tarcieri: yes. |
| 05:48:16 | evan | probably. |
| 05:48:28 | evan | but we copied strongtalk |
| 05:48:33 | evan | which is prior art. |
| 05:48:36 | evan | so fuck 'em. |
| 05:48:38 | tarcieri | nice! |
| 05:48:39 | tarcieri | heh |
| 05:49:19 | tarcieri | it now makes sense to me why the .NET CLR doesn't have an interpreter |
| 05:49:59 | tarcieri | how the hell can you patent something that abstract though? jeezus |
| 05:51:01 | brixen | did someone patent the process of patenting? |
| 05:51:02 | scoopr | don't all the new javascript engines do basically the same as well? |
| 05:51:07 | evan | brixen: yes |
| 05:51:11 | tarcieri | scoopr: V8 doesn't |
| 05:51:16 | tarcieri | scoopr: but yeah, a bunch of the other ones do |
| 05:51:22 | evan | someone patenting using a patent as a 2nd party against someone else |
| 05:52:13 | bakkdoor | lol hopefully we're not getting software patents in the eu |
| 05:52:25 | bakkdoor | good evening btw :P |
| 05:52:33 | brixen | hi bakkdoor |
| 05:52:36 | tarcieri | heh |
| 05:52:41 | tarcieri | yeah that's really weird with my company |
| 05:52:45 | tarcieri | since we're spread all over the world |
| 05:52:46 | scoopr | I wonder if it's good business to software "us-excluding" :P |
| 05:52:50 | tarcieri | a bunch of patents only apply in the US |
| 05:52:51 | bakkdoor | hi brixen :) |
| 05:52:51 | scoopr | +do |
| 05:53:08 | tarcieri | scoopr: a ton of patents on the stuff we do just disappears when you're not dealing with the US market |
| 05:53:49 | bakkdoor | true.. although there was a court case recently in germany that kind of made software patents legal.. we'll see how that progresses.. |
| 05:54:37 | tarcieri | oh yeah, here's my personal "most ridiculous usage of git ever": http://github.com/tarcieri/resque/commit/2e66384b750d6923c673cab0f57e9b88424cd6a7 |
| 05:54:50 | tarcieri | defunkt requested a patch, and I'm like "hey it's github, might as well use it" |
| 05:57:01 | bakkdoor | haha nice :P |
| 05:57:38 | tarcieri | it's pretty badass the json gem is now portable |
| 05:57:43 | bakkdoor | which reminds me to get back to working on fancy again once i'm back in germany. haven't had time to work on it for the whole time i've been here really |
| 05:57:47 | bakkdoor | sounds good |
| 05:58:01 | tarcieri | took a lot of bitching on my part :) |
| 05:58:15 | bakkdoor | ^^ |
| 05:59:19 | tarcieri | evan: do you have a whole point of "irreducible complexity" when it comes to inlining? |
| 05:59:27 | tarcieri | I only have the vaguest idea of how that works on HotSpot |
| 06:00:05 | evan | you mean when trying to determine what to inline? |
| 06:00:21 | tarcieri | well, more like when to inline, I guess |
| 06:00:34 | evan | well, when to inline is a fucking guess |
| 06:00:36 | evan | everyone guesses. |
| 06:00:37 | tarcieri | heh |
| 06:00:52 | bakkdoor | guesses are cool |
| 06:01:02 | evan | no one has figured out a cheap algorithm for determining an optimal when |
| 06:01:22 | evan | what to inline optimal is irreducible pretty much |
| 06:01:28 | tarcieri | yeah, HotSpot apparently has some threshhold of code complexity at which point it's like "fuck that, I can't help you, sorry" |
| 06:01:38 | evan | you have to build a massive lattice of values and even then, it's a trade off. |
| 06:02:13 | bakkdoor | it probably is undecidable within a decent timeframe.. stochasticly determining it probably is the best way to go |
| 06:02:24 | evan | well, hotspot gives up if a method gets too big |
| 06:02:32 | tarcieri | evan: yeah that's what I'm talking about |
| 06:02:53 | evan | there is no specific reason to do that other than giant methods hang up the compiler thread |
| 06:03:08 | evan | they could still use cheap inline decisions |
| 06:04:15 | tarcieri | wouldn't inlining naturally result in big complex methods? |
| 06:04:29 | evan | it inflates them, yes. |
| 06:04:35 | evan | but you can govern that |
| 06:04:37 | evan | i do. |
| 06:04:52 | evan | far from perfectly, but you can. |
| 06:05:16 | tarcieri | I really have no idea what you're doing but it sounds pretty exciting :) |
| 06:05:54 | evan | i'm just guessing. |
| 06:06:08 | evan | everyone i've talked to about dynamic compilers makes it up. |
| 06:06:10 | evan | metric wise. |
| 06:06:27 | evan | "umm, how about 5123 runs until compiled" "sure, why not." |
| 06:06:54 | tarcieri | hehe |
| 06:08:50 | evan | i need to do more stats work on that number and figuring out what methods to inline |
| 06:09:01 | evan | atm, i just inline like crazy until the budget is exhausted. |
| 06:10:01 | evan | finding out what other runtimes do is... hard. |
| 06:10:11 | evan | that logic is buried deep deep. |
| 06:21:25 | tarcieri | heh |
| 06:21:32 | tarcieri | yeah sounds like voodoo |
| 07:01:24 | dbussink | morning |
| 07:02:09 | dbussink | evan: if you're interested, this is a backtrace with current hydra i have now: https://gist.github.com/46e4061f474ccc27d49e |
| 07:14:12 | postmodern | man brixen wasn't joking |
| 07:14:22 | postmodern | the rubinius jit is *fast* |
| 07:18:01 | dbussink | postmodern: of course it is :) |
| 07:18:10 | postmodern | dbussink, and how |
| 07:18:15 | dbussink | postmodern: what are you running? |
| 07:18:29 | postmodern | dbussink, i double checked the numbers to make sure it wasn't just startup time |
| 07:18:38 | postmodern | dbussink, first gen amd64 |
| 07:18:41 | postmodern | dbussink, most laptops are faster than my desktop now |
| 07:18:53 | dbussink | postmodern: i mean what ruby code :) |
| 07:19:00 | postmodern | dbussink, also mri 1.9.2, rubinius head, python 2.6 |
| 07:19:08 | postmodern | dbussink, just fib code under time |
| 07:19:22 | dbussink | ah, the infamous fib benchmark :P |
| 07:19:32 | dbussink | totally useless but fun to show ;) |
| 07:19:34 | postmodern | yep tail-recursion benchs |
| 07:19:43 | postmodern | well recursion took to the extreme |
| 07:20:18 | postmodern | i wonder how LLVM handles algebraic optimizations? |
| 07:20:29 | postmodern | like expanding x ** 2 to x * x |
| 07:21:00 | postmodern | or using shl shr instead of x * 2 or x / 2 |
| 07:28:27 | dbussink | postmodern: dunno how llvm does those exactly |
| 07:28:35 | dbussink | postmodern: but those aren't what makes something like fib fast |
| 07:28:58 | postmodern | dbussink, but it helps for number heavy apps |
| 07:29:12 | postmodern | dbussink, since pow's and multi's can add up |
| 07:29:26 | dbussink | postmodern: true, but the effects are usually relatively minimal |
| 07:29:26 | postmodern | dbussink, for things like imagine processing or crypto |
| 07:29:38 | dbussink | yeah, there it can help |
| 07:29:39 | postmodern | dbussink, ha idk |
| 07:29:44 | postmodern | yeah |
| 07:29:57 | postmodern | dbussink, that' the algorithm taken to the extreme |
| 07:30:09 | postmodern | dbussink, much like fib, it has a certain property that is maxed out |
| 07:31:04 | postmodern | dbussink, also most of what makes Java smoking fast in number crunching demos are tons of backhanded tricks |
| 07:31:29 | postmodern | dbussink, that and all the money thrown at it |
| 07:34:06 | postmodern | dbussink, hmm not looking good for i ** 2 in Rubinius |
| 07:34:23 | postmodern | dbussink, mri 1.9.2-rc2 finished around 1min |
| 07:34:31 | postmodern | dbussink, rbx i had to stop after 5min |
| 07:34:42 | dbussink | postmodern: well, first of all, i ** 2 doesn't mean i * i in ruby perse |
| 07:34:55 | postmodern | dbussink, it doesn't but it's equivalent for Integers |
| 07:34:56 | dbussink | because the ** method can do something totally different |
| 07:35:06 | postmodern | dbussink, unless it's not overrided |
| 07:35:07 | dbussink | true, but people can overwrite that for integers |
| 07:35:15 | postmodern | but if they dont |
| 07:35:22 | postmodern | bonus points in the performance department |
| 07:35:28 | postmodern | also why is MRI faster? |
| 07:35:35 | dbussink | but you have to check whether it's the dispatching that is most overhead or the actual implementation of ** |
| 07:35:42 | postmodern | i consider this a longterm performance issue? |
| 07:36:06 | dbussink | well, i haven't seen ** come up in any performance profile, so it's not an issue |
| 07:36:19 | postmodern | dbussink, puts (1..100_000_000).inject { |sum,i| sum + i ** 2 } |
| 07:36:26 | dbussink | so if you have real code that depends on this performance, please generate some benchmarks |
| 07:36:39 | dbussink | postmodern: well, we have a bunch benchmarks in there already |
| 07:36:49 | dbussink | postmodern: you can run those and see if you can optimize them if you want |
| 07:36:56 | postmodern | dbussink, and how long will i have to wait for it to be fixed, so my real code can be fast? :) |
| 07:37:03 | dbussink | there are other things way more important things than that |
| 07:37:09 | postmodern | ok |
| 07:37:14 | dbussink | most likely until you fix it yourself ;) |
| 07:37:23 | postmodern | so any ** related code in Rubinius *will* be much slower than mri |
| 07:37:27 | postmodern | duely noted |
| 07:37:33 | dbussink | most likely not |
| 07:37:41 | dbussink | since nothing does so much ** that it will be noticed |
| 07:37:43 | postmodern | do not use rubinius for power related computations |
| 07:38:01 | dbussink | it being a tad slower, but that is by far overwhelmed by other code being run |
| 07:38:22 | dbussink | we can make ** faster, or perhaps we can make method dispatch faster |
| 07:38:23 | postmodern | dbussink, unless all your doing is number crunching |
| 07:38:31 | postmodern | dbussink, than Rubinius is a horrible choice |
| 07:38:57 | postmodern | dbussink, let me try dispatching a simple method call and comparing times |
| 07:38:57 | dbussink | postmodern: depends on what kind of number crunching |
| 07:39:11 | postmodern | dbussink, number crunching involving complex operators of course |
| 07:39:27 | dbussink | because a lot of stuff like +, -, * and / is faster |
| 07:39:45 | postmodern | it is |
| 07:39:45 | dbussink | postmodern: if you depend on number crunching performance like that, you usually don't use ruby at all |
| 07:39:47 | postmodern | er |
| 07:39:58 | postmodern | well shl and shr will always be faster than * 2 or / 2 |
| 07:40:08 | postmodern | dbussink, that's defeatist thinking sir |
| 07:40:09 | dbussink | but some statically typed language that always will be faster |
| 07:40:18 | postmodern | dbussink, since MRI *is* faster in this case |
| 07:40:20 | dbussink | postmodern: that's realistic thinking :) |
| 07:40:42 | dbussink | maybe mri has some peephole optimizations? |
| 07:40:47 | dbussink | did you check that? |
| 07:40:53 | postmodern | probably maybe |
| 07:40:56 | postmodern | i didn't |
| 07:41:02 | postmodern | im just illuminating this issue |
| 07:41:17 | postmodern | giving feedback almost |
| 07:41:22 | dbussink | well, the problem is that there are probably thousands of these issues :) |
| 07:41:23 | postmodern | about Rubinius's VM/JIT |
| 07:41:43 | postmodern | ok i changed the benchmark to i * i |
| 07:41:44 | dbussink | well, this is why benchmarking is really really hard |
| 07:41:56 | postmodern | and it shaved off 20s |
| 07:41:56 | dbussink | because people will jump to generic conclusions |
| 07:42:05 | postmodern | so your probably right in method dispatch overhead |
| 07:42:30 | postmodern | but still, if MRI could save 20s, rubinius could also |
| 07:43:01 | postmodern | also these optimizations are pretty old tricks |
| 07:43:14 | dbussink | well, nothing is stopping you from trying to add them :) |
| 07:43:26 | postmodern | well first i have to finish my time-machine |
| 07:43:31 | dbussink | postmodern: the point is that there are way more important things to improve right now :) |
| 07:43:32 | postmodern | so i can add extra hours to my work day |
| 07:43:45 | postmodern | once that's done, i'll get those patches submitted and documented ASAP |
| 07:43:53 | postmodern | dbussink, indeed indeed |
| 07:44:08 | postmodern | dbussink, just pointing out some low hangs fruit |
| 07:44:15 | postmodern | *low hanging fruit |
| 07:45:04 | dbussink | postmodern: well, we all have limited resources too :) |
| 07:45:13 | postmodern | ha |
| 07:45:22 | postmodern | dbussink, i'll share my time-machine with you when i complete it |
| 07:45:22 | dbussink | for example string performance is way more important atm |
| 07:45:30 | dbussink | because that affects way more real life code out there |
| 07:45:39 | postmodern | oh double-word (no-pun) to string performance |
| 07:45:45 | postmodern | how is Array perf too? |
| 07:45:55 | dbussink | so unless someone has a pressing need to improve **, other things will happen first |
| 07:46:37 | dbussink | postmodern: you can try to run the available benchmarks |
| 07:46:45 | dbussink | postmodern: there are a bunch for various objects |
| 07:46:48 | dbussink | classes |
| 07:46:53 | postmodern | cool |
| 16:21:57 | evan | morning |
| 16:22:18 | evan | dbussink: i really don't like "00:38 postmodern: dbussink, than Rubinius is a horrible choice" |
| 16:22:21 | evan | :( |
| 16:29:35 | brixen | morning |
| 16:32:21 | evan | hello sir! |
| 17:16:19 | dbussink | evan: yeah, well, i try not to get too fussed up about stuff like that |
| 17:19:42 | brixen | dbussink: well, his point was that rbx is slower on something and I don't think the response is "oh well, it's not used that much" |
| 17:19:59 | brixen | we can just ask for code or benchmarks and see what we can do to make it faster |
| 17:20:11 | dbussink | yeah, well, we already have those benchmarks |
| 17:20:15 | brixen | I fully expect to use rbx for "number crunching" code |
| 17:20:37 | dbussink | just tried to make clear that we can't jump on just any of such reports |
| 17:20:59 | brixen | well, give us a report and we'll see if we can jump on it :) |
| 17:21:21 | dbussink | well, i think string in general can use some love ;) |
| 17:21:33 | brixen | ETOOGENERAL |
| 17:21:43 | dbussink | hehe |
| 17:21:46 | brixen | String has had lots of love |
| 17:22:00 | evan | needs love is not a valid bug report. |
| 17:22:10 | evan | data talks. |
| 17:22:23 | evan | :) |
| 17:22:25 | brixen | needing love in general is not a bug :) |
| 17:22:30 | brixen | everyone needs love |
| 17:22:48 | dbussink | group hug! |
| 17:22:52 | brixen | heh |
| 17:23:04 | dbussink | evan: i have a hydra crash for you if you want data ;) |
| 17:23:05 | evan | skips around the meadow |
| 17:23:10 | evan | i saw. |
| 17:23:15 | evan | it's sadly not useful |
| 17:23:33 | evan | because it's a crash inside LLVM |
| 17:23:36 | evan | so there is no place to start. |
| 17:25:14 | dbussink | evan: ah, too bad |
| 17:25:32 | dbussink | evan: because something else corrupted code then that llvm chokes on subsequently? |
| 17:25:45 | evan | has to be |
| 17:25:49 | evan | LLVM itself can't crash. |
| 17:26:04 | evan | so if you see a crash inside it, thats not LLVMs fault. |
| 17:26:15 | evan | and because we use in fairly opaquely |
| 17:26:23 | evan | tracking a crash from inside it isn't useful. |
| 17:26:44 | dbussink | evan: and llvm itself should still run single threaded right? in the compiling thread? |
| 17:27:02 | evan | yeah, LLVM runs serially |
| 17:27:05 | evan | in it's own thread |
| 17:27:08 | evan | as it always has. |
| 17:27:13 | evan | no change there. |
| 17:29:39 | dbussink | evan: ran clean through a ci run now though :) |
| 17:29:47 | evan | yeah, so there is a bug somewheres |
| 17:29:47 | brixen | woot! |
| 17:29:57 | dbussink | evan: going to try if compiling specs makes a difference |
| 17:30:03 | dbussink | if it has to do that or now |
| 17:30:04 | dbussink | not |
| 17:30:06 | evan | we just have to coax it out in a way that lets us see it. |
| 17:30:26 | evan | a crash inside LLVM typically means something used some memory that was freed |
| 17:30:37 | evan | ie, we corrupted some of LLVMs memory |
| 17:39:15 | dbussink | evan: haven't been able to get it to crash again now though |
| 17:39:28 | evan | welcome to thread bugs. |
| 17:39:44 | dbussink | yeah, always annoying |
| 17:40:18 | dbussink | evan: i'm going to torture it with datamapper specs |
| 17:40:31 | evan | ok |
| 17:40:41 | dbussink | see if it buckles under that |
| 17:47:08 | dbussink | evan: i should write some torture script that is doing nasty stuff in parallel threads :) |
| 17:47:17 | evan | do it. |
| 17:47:22 | evan | thats how i've been finding stuff |
| 17:47:28 | evan | the last round was doing |
| 17:47:38 | evan | 100000.times { timeout(1) { 1 + 1 } } |
| 17:47:46 | evan | and fixing the bugs that that created. |
| 17:47:56 | evan | s/created/exposed/ |
| 17:59:41 | Zoxc | LLVM is totally bug free :] |
| 18:02:22 | evan | for this experiment, it is. |
| 18:03:11 | sbryant | You do not question the LLVM. |
| 18:03:21 | sbryant | You do not taunt LLVM. |
| 18:05:43 | dbussink | evan: hmm, my torture tests seem to happily create threads and go along |
| 18:06:04 | evan | :D :D |
| 18:06:39 | dbussink | 1.8's green threads are cheap though |
| 18:07:02 | dbussink | looking at the performance |
| 18:08:09 | dbussink | but rbx seems way faster on this thread creation than 1.9 for me |
| 18:11:38 | evan | nice! |
| 18:11:53 | evan | we'll probably eventually have to add some thread pooling |
| 18:12:08 | evan | a small pool of 3 or 4 threads would probably be fine. |
| 18:23:28 | dbussink | evan: to spin up threads fast you mean? by reusing them from a pool? |
| 18:23:34 | dbussink | evan: jvm does stuff like this too right? |
| 18:24:15 | dbussink | evan: 1.9 seems to crumble with large numbers of threads, rbx scales up linearly at least until creating 100000 threads :) |
| 18:24:19 | evan | yeah |
| 18:24:28 | evan | zoinks! |
| 18:24:31 | evan | 100000!!! |
| 18:26:34 | dbussink | evan: takes around 16 seconds |
| 18:26:44 | dbussink | also runs some bits of code :P |
| 18:26:50 | dbussink | i didn't want for 1.9 to finish |
| 18:26:57 | evan | did you collect the threads into an array? |
| 18:27:02 | dbussink | evan: yeah |
| 18:27:04 | evan | or did you allow them to be GCd |
| 18:27:05 | evan | k |
| 18:27:09 | dbussink | evan: and joined on all them |
| 18:27:29 | evan | rbx wise, they'll run and exit themselves before others start |
| 18:27:34 | evan | so you'll get a window |
| 18:27:39 | evan | because of the concurrency |
| 18:27:42 | evan | where as with 1.9 |
| 18:27:54 | evan | you won't because the GIL prevents the windowing effect |
| 18:28:13 | evan | so with rbx you'll get maybe 50 threads running concurrently, but with 1.9 you'll get thousands and thousands |
| 18:28:37 | blowmage | evan: windowing effect? |
| 18:29:12 | evan | blowmage: when the threads are concurrent |
| 18:29:27 | evan | by the time the 51st thread is spun up, the 1st thread has already exited |
| 18:29:29 | dbussink | evan: got a crash :) |
| 18:29:44 | dbussink | evan: ah, ok, that makes sense yeah |
| 18:29:49 | evan | so the kernel will reuse resources from the 1st thread in the 51st thread |
| 18:30:17 | evan | the more cores you have, the bigger the window |
| 18:30:17 | dbussink | ok, this was dumb |
| 18:30:23 | evan | hah |
| 18:30:25 | dbussink | thread apply all bt |
| 18:30:27 | evan | did you swamp your machine? |
| 18:30:28 | evan | haha |
| 18:30:29 | evan | oops. |
| 18:30:30 | dbussink | in my gdb crash |
| 18:31:32 | dbussink | evan: Assertion failed: (refill_slab((*i)->local_slab())), function collect_young, file vm/objectmemory.cpp, line 192 |
| 18:31:38 | dbussink | i have 100 threads now |
| 18:31:39 | evan | fun. |
| 18:31:49 | evan | thats weird.. |
| 18:32:01 | evan | line 192? |
| 18:32:08 | evan | :/ |
| 18:32:12 | evan | there is no assert there.. |
| 18:32:56 | dbussink | evan: this is the backtrace for the crash for the thread in which it occured: https://gist.github.com/5643629c407601f7f297 |
| 18:34:15 | dbussink | evan: ah, wait, sorry |
| 18:34:45 | dbussink | i think my build is a bit confused about master / hydra atm |
| 18:34:48 | dbussink | let me recheck |
| 18:34:53 | evan | k |
| 18:35:11 | evan | thats definitely something to verify. |
| 18:35:40 | dbussink | maybe this even crashes master |
| 18:35:49 | evan | could be. |
| 18:41:29 | dbussink | evan: what's your idea on behavior of stuff like Array#<< when there's no gil? |
| 18:42:15 | evan | headius and I have discussed that. |
| 18:42:37 | evan | basically, it's not possible to make the core classes like String/Array thread safe and keep them at their current performance level |
| 18:43:04 | evan | so they can tolerate internal confusion, but they can't cause the VM to crash. |
| 18:43:07 | dbussink | evan: this is same test under hydra: https://gist.github.com/1c08e79088ee558f1854 |
| 18:43:29 | evan | interesitng. |
| 18:43:30 | dbussink | 1000 threads, so i can sample a few, but listing all is tricky :) |
| 18:43:32 | evan | ok |
| 18:43:39 | evan | i know what the issue is. |
| 18:43:48 | evan | you need more slabs that the eden can provide |
| 18:43:58 | evan | master has the same issue. |
| 18:44:08 | dbussink | ah, exhausting eden actually |
| 18:44:12 | dbussink | because of the number of threads |
| 18:44:15 | evan | yep. |
| 18:44:26 | evan | because each thread wants at least one slab |
| 18:44:39 | dbussink | yeah |
| 18:44:48 | evan | what we should do it just set the slab size to 0 |
| 18:45:02 | evan | so that all allocations in that thread are slower, but capable. |
| 18:45:06 | evan | feel free to make that change. |
| 18:45:11 | evan | rather than the assert, do |
| 18:45:16 | dbussink | evan: you mean if it can't allocate a slab? |
| 18:45:32 | evan | if(addr) { slab.refill(addr, slab_size_); } else { slab.refill(0, 0); } |
| 18:45:51 | evan | right, if there are no slabs to have |
| 18:45:59 | evan | refill the local slab with a size of 0 |
| 18:46:03 | evan | so that using it always fails |
| 18:46:05 | evan | at runtime |
| 18:46:28 | evan | and goes out to the normal ObjectMemory allocations |
| 18:47:51 | dbussink | evan: what does the objects_allocated += slab.allocations(); do there? |
| 18:48:09 | evan | so, each slab tracks how many times it was used |
| 18:48:10 | dbussink | does that need to happen in that case too? |
| 18:48:20 | evan | that should happen no matter what, yeah. |
| 18:48:33 | evan | basically, we keep metrics on allocations now |
| 18:48:41 | evan | and to keep that metric thread safe |
| 18:48:45 | evan | the slab tracks it's own allocations |
| 18:48:58 | evan | and all slabs allocations are added to the global total each GC |
| 18:49:21 | dbussink | evan: ah ok, makes sense then yeah |
| 18:49:30 | dbussink | and this gets the allocations before the slab was refilled |
| 18:50:26 | evan | yes. |
| 18:50:34 | evan | refill resets allocations to 0 |
| 18:50:41 | dbussink | i assumed as much :) |
| 18:52:02 | dbussink | evan: do appreciate explaining stuff like this :) |
| 18:52:11 | evan | no problem! |
| 18:52:24 | dbussink | makes total sense now, but don't have that high level picture you have in your head that makes it easy to pinpoint stuff :) |
| 18:52:28 | evan | feel free to add a comment in there that reflects what i told ya also. |
| 18:52:38 | dbussink | yeah, already did add some :) |
| 18:53:19 | evan | k :) |
| 18:54:59 | dbussink | evan: onto the next one! |
| 18:55:14 | evan | did ya push it? |
| 18:55:24 | dbussink | evan: https://gist.github.com/051776023f0c0e445ce5 |
| 18:55:25 | dbussink | evan: not yet |
| 18:55:50 | evan | k |
| 18:58:45 | boyscout | Fallback to ObjectMemory allocation if a slab can't be allocated - ff3dedc - Dirkjan Bussink (hydra) |
| 19:03:03 | dbussink | evan: any idea what could cause that one? |
| 19:03:40 | evan | looks like perhaps the global list of threads is getting corrupted |
| 19:03:56 | evan | go through everywhere it's used and make sure they're all syncronized |
| 19:04:26 | evan | you don't need to worry about synchronizing reading it in the GC btw |
| 19:05:12 | dbussink | evan: the gc is locked yeah |
| 19:05:18 | evan | right |
| 19:06:30 | dbussink | evan: hmm, stop_the_world runs already locked? |
| 19:06:41 | evan | in a way, yes. |
| 19:06:50 | evan | it's more complicated than a single lock. |
| 19:07:16 | evan | because it waits for all other threads to checkin before continuing |
| 19:08:22 | dbussink | evan: could it be the agent? |
| 19:08:58 | evan | mm |
| 19:09:00 | evan | i don't think so |
| 19:09:05 | dbussink | it has GlobalLock::LockGuard uncommented |
| 19:09:06 | evan | unless you were using the agent |
| 19:09:11 | dbussink | but i'm not using the agent |
| 19:09:14 | evan | yeah, but you didn't have the agent on |
| 19:09:18 | evan | so it could'nt be the agent. |
| 19:10:57 | dbussink | evan: because i can't find it used unlocked or outside gc anywhere else |
| 19:11:39 | evan | it being the list? |
| 19:11:48 | dbussink | evan: the threads list yeah |
| 19:11:52 | evan | hm |
| 19:11:58 | evan | ok, well, you'll have to dig around a little more. |
| 19:13:00 | dbussink | evan: hmm, maybe SharedState itself? |
| 19:13:07 | dbussink | that it's refcounts are wrong? |
| 19:13:56 | evan | *shrug* |
| 19:13:59 | evan | thats for you to find out. |
| 19:14:22 | dbussink | i'll poke a bit in the dark then :p |
| 19:14:36 | evan | thats what I do |
| 19:14:39 | evan | if you can repro it easily |
| 19:14:43 | evan | then try some different things |
| 19:32:45 | boyscout | Use a channel to coordinate Timeout better - e473723 - Evan Phoenix (hydra) |
| 19:32:46 | boyscout | Add the semaphore count optimization to Channel - 44ba8d1 - Evan Phoenix (hydra) |
| 19:32:46 | boyscout | Use a dedicated place to store the interrupting exception - fd708d8 - Evan Phoenix (hydra) |
| 20:04:05 | dbussink | evan: hmm, could it be related to joining on an already closed thread? |
| 20:30:56 | dbussink | evan: still there? |
| 21:19:36 | evan | hey |
| 21:19:43 | brixen | yo |
| 21:20:04 | brixen | now we just need the rest of the sentence |
| 21:25:32 | evan | heh |
| 23:22:03 | idl | hey, what's this hydra branch all about? I'm intrigued |
| 23:22:55 | idl | that topic is amazing |
| 23:25:02 | brixen | idl: it's a branch to work on removing the GIL |
| 23:25:16 | idl | ah cool |
| 23:25:22 | brixen | yes :) |
| 23:32:16 | evan | brixen: so, we setup the active_path of required files to be their expanded path |
| 23:32:47 | evan | rather than simple load_path_entry + subpath |
| 23:32:52 | evan | is there a reason for that? |
| 23:34:05 | evan | erm, maybe not... hm... |
| 23:37:31 | brixen | active_path should be what require was given iirc |
| 23:37:44 | evan | yeah, i'm investigating |
| 23:40:39 | evan | oh i see! |
| 23:40:42 | evan | doing -Ilib |
| 23:40:53 | evan | is putting File.expand_path(part) into $: |
| 23:41:21 | evan | and it's not suppose to expand it |
| 23:41:36 | evan | which is dumb, because if you chdir, that lib is something else then |
| 23:41:38 | evan | *eyeroll* |
| 23:48:43 | brixen | yes, that's a quite crazy situation |