Da
Show enters and exits. Hide enters and exits.
| 00:00:01 | slava | the dispatch is eliminated in foo above because it knows the indices are fixnums, and pi is a float |
| 00:00:08 | evan | if i don't retain all the Floats |
| 00:00:12 | evan | but still create them |
| 00:00:19 | slava | it still boxes the final result, and does a gc ceheck on every iteration |
| 00:00:26 | evan | it's at 1.3s |
| 00:00:36 | slava | hmm |
| 00:00:39 | evan | so the vast, vast majority of time is GC. |
| 00:00:42 | slava | yep |
| 00:01:15 | slava | Number Total Mean Median Min Max |
| 00:01:15 | slava | Copying from nursery 77 117,837 µs 1,530 µs 1,993 µs 11 µs 2,452 µs |
| 00:01:18 | slava | Copying from aging 31 367,265 µs 11,847 µs 10,167 µs 3,065 µs 16,065 µs |
| 00:01:21 | slava | Copying to tenured 1 10,937 µs 10,937 µs 10,937 µs 10,937 µs 10,937 µs |
| 00:01:35 | slava | are you seeing a similar number of GC passes I wonder? |
| 00:02:51 | evan | hm, i should add a GC summary to -Xgc.show |
| 00:03:39 | evan | 165 young |
| 00:03:47 | evan | 19 old |
| 00:03:53 | evan | well, 19 full |
| 00:03:56 | evan | collections. |
| 00:04:02 | slava | how long is each one? |
| 00:04:21 | evan | the last young is 62ms |
| 00:04:38 | evan | the last full was 52ms |
| 00:04:44 | slava | whoa, weird |
| 00:04:47 | evan | er wait |
| 00:04:49 | evan | wrong side |
| 00:04:51 | slava | full is faster tha nyoung? |
| 00:04:53 | evan | last full was 222ms |
| 00:05:12 | slava | ok |
| 00:05:35 | slava | how much live data? |
| 00:05:50 | evan | if i use a seperate remember set that has address-of-slot rather than address-of-object for Tuples |
| 00:05:55 | evan | i think it will speed up. |
| 00:06:12 | slava | how much data do you manage to scan in 222ms? |
| 00:06:22 | evan | lets see. |
| 00:06:51 | evan | 345M |
| 00:07:01 | slava | impressive |
| 00:07:51 | slava | my mark/sweep takes 200ms to collect a heap with 47mb live |
| 00:08:03 | slava | does your full gc not scan every object or something? |
| 00:08:05 | slava | is it incremental? |
| 00:08:10 | evan | it scans them all |
| 00:08:18 | evan | actually wait. |
| 00:08:30 | evan | i grabed the total heap number |
| 00:08:36 | evan | [Full GC 203238kB => 175556kB 216ms] |
| 00:08:48 | evan | so that was 216ms to scan 200M |
| 00:09:33 | slava | with compaction, 380ms to do a full gc on the same data, 47mb data + 7mb code |
| 00:09:43 | slava | seems a lot slower than yours |
| 00:09:50 | evan | thats not doing compaction |
| 00:09:56 | evan | but even if it were |
| 00:09:59 | slava | are your objects mostly pointers or mostly empty space / binary data? |
| 00:10:02 | evan | the compaction is folded into the next mark phase |
| 00:10:07 | evan | thats the beauty of immix. |
| 00:10:31 | slava | is the mark phase pretty conventional, with a mark stack and iteration over all slots? |
| 00:10:37 | evan | yep. |
| 00:10:57 | slava | see my GC spends most of its time in the mark phase |
| 00:11:18 | evan | my mark phase uses virtual methods even. |
| 00:11:31 | slava | heh |
| 00:11:53 | evan | because it lets the object (type) control it's own layout |
| 00:11:54 | slava | what's your secret sauce |
| 00:12:26 | evan | well, lets see, it's pretty basic. |
| 00:12:37 | evan | the mark stack is a std::vector<> still |
| 00:12:41 | slava | same |
| 00:12:55 | evan | are you pushing/poping from the back? |
| 00:13:08 | slava | I iterate over slots in a somewhat generic way |
| 00:13:20 | slava | I bet you just have a visitor method for each type that does something with every slot? |
| 00:13:24 | slava | without the loop? |
| 00:13:50 | evan | basically, yes |
| 00:13:55 | evan | the visitor is dispatched as a virtual method. |
| 00:14:16 | evan | and it's passed an ObjectMark& object |
| 00:14:34 | evan | and the method uses mark.call() to perform the actual marking |
| 00:14:42 | evan | so the marking is abstracted from the visitor |
| 00:15:48 | slava | wow |
| 00:20:33 | maharg | is rubinius' standard build procedure actually using llvm-clang to compile now? |
| 00:20:51 | evan | no |
| 00:20:52 | evan | never has. |
| 00:21:08 | boyscout | Fix JIT code unloading - fd84366 - Evan Phoenix |
| 00:21:15 | maharg | oh I see |
| 00:21:24 | maharg | the build step that was confusing me was building llvm itself |
| 00:22:50 | evan | you shouldn't have to do that either |
| 00:22:56 | evan | unless there is no prebuilt for your platform. |
| 00:23:14 | maharg | apparently there wasn't, I just did ./configure && rake |
| 00:23:36 | maharg | osx 10.5.7 (I'm a little behind on this laptop) |
| 00:23:41 | evan | what platform are you on? |
| 00:23:42 | evan | hm |
| 00:23:46 | evan | there is one for 10.5 |
| 00:23:58 | evan | run ./rakelib/config.guess |
| 00:24:00 | evan | what is the output? |
| 00:24:30 | maharg | i686-apple-darwin9.7.0 |
| 00:25:43 | evan | hm, 9.8.0 is whats up there |
| 00:25:46 | evan | let me link it |
| 00:25:47 | evan | and you can try it. |
| 00:26:10 | evan | ok, cancel that and do |
| 00:26:15 | evan | rm -rf vm/external_libs/llvm |
| 00:26:17 | evan | ./configure |
| 00:28:04 | maharg | used a binary package, is currently building rubinius |
| 00:28:08 | evan | k |
| 00:28:11 | boyscout | Fix docs a little - 8eb566a - Evan Phoenix |
| 00:29:19 | boyscout | CI: rubinius: fd84366 successful: 3024 files, 11752 examples, 35958 expectations, 0 failures, 0 errors |
| 00:36:17 | maharg | tests seem to be running fine |
| 00:37:37 | evan | maharg: cool. |
| 00:39:49 | maharg | 0 fail 0 errors |
| 00:42:53 | evan | well thats good! |
| 00:47:46 | maharg | incidentally, what's the current state of threading within the VM? Last time I was paying attention it was still all green-threaded I think. With big plans, but still for a 1.9-style BGL iirc? |
| 00:48:38 | evan | well now |
| 00:48:46 | evan | you haven't been around for the last 1+ year i'm assume |
| 00:48:49 | evan | it's totally different. |
| 00:49:10 | evan | it's using native threads with a GIL right now |
| 00:49:20 | evan | we're starting to lay the ground work for removing the GIL |
| 00:49:25 | evan | but that probably won't happen before 1.0 |
| 00:50:46 | maharg | cool |
| 00:51:09 | maharg | huh. jruby moved from sun to engineyard? That's surprising |
| 00:51:32 | evan | a while ago, yeah |
| 01:00:11 | jarib | evan: how will that work with C extensions? |
| 01:00:24 | evan | we haven't worked it out yet. |
| 01:01:06 | jarib | gotcha |
| 01:01:36 | jarib | isn't that the reason YARV still has it? |
| 01:01:44 | evan | no |
| 01:01:49 | evan | well, not entirely |
| 01:02:09 | evan | MRI (and thusly YARV) use the same API internally as the C extensions do |
| 01:02:25 | evan | they'd have to add locks and basically make MRI's core thread-safe |
| 01:02:34 | evan | they've done some of that legwork already |
| 01:02:34 | jarib | ah, makes sense |
| 01:02:49 | evan | but the shear amount of C code to fix is the issue. |
| 01:03:40 | maharg | In broad strokes, I would have extensions advise as to their thread safety and put locks around their calls if they don't claim to be threadsafe. And have a switch to force all extensions to go through that lock in case you run into a liar. |
| 01:03:57 | evan | that requires changes to all extensions |
| 01:04:04 | evan | some that probably won't fly very well. |
| 01:04:19 | evan | we insulate extensions from the VM already |
| 01:04:21 | evan | behind handles |
| 01:04:41 | evan | so it's likely we can use the handles to provide at least some thread safety |
| 01:05:35 | evan | certainly using a global lock around all extension calls is one solution |
| 01:06:12 | maharg | well, I would assume any ext that can't claim to be threadsafe isn't. With cooperation with the MRI guys, that should be possible |
| 01:06:35 | maharg | even if it took them a lot longer to get there |
| 01:07:26 | evan | maharg: i'd like to think they'd want to help, but that hasn't proved to be true thus far. |
| 01:07:40 | evan | they're priorities doen't line up with ours often. |
| 01:07:45 | evan | their, rather. |
| 01:10:13 | evan | but again, who knows. |
| 01:10:19 | maharg | I like the new C++ code. Very clean. And you guys didn't reinvent a bunch of perfectly good wheels (like vector, for example) |
| 01:10:33 | evan | 1.9 has started to push on this idea because of MVM |
| 01:10:35 | maharg | as is quite common in projects transitioning from C |
| 01:10:41 | evan | yeah, ew. |
| 01:10:45 | evan | thats one reason we switched |
| 01:10:57 | evan | not having to maintain or own basic datastructures. |
| 01:11:09 | evan | s/or/our/ |
| 01:14:04 | Zoxc | doesn't like the C++ STL datastructures :( |
| 01:16:06 | boyscout | Fix format cases with % u and %+u. Fixies #162 - eb97899 - Ivan Samsonov |
| 01:16:06 | boyscout | Add String#to_inum support for soft bases. Fixes #163. - 398d64d - Evan Phoenix |
| 01:16:16 | maharg | they're cleaner, better performing, more predictable, and better speced than the vast majority of roll-your-own equivalents. |
| 01:16:22 | evan | yeah |
| 01:16:26 | evan | they're battle hardened. |
| 01:16:29 | evan | for sure. |
| 01:16:58 | evan | having C++ classes that mirror the ruby classes also has simplified the code base |
| 01:17:18 | evan | and a little meta-magic to attach a C++ method to a ruby method, and things are easily up and running. |
| 01:18:51 | maharg | speaking of MVM, is there a reference to the rubinius implementation somewhere? I see mentions of it all over google, but nothing very specific |
| 01:19:19 | maharg | heh I don't think docs/virtual_machine.txt has changed since I last looked |
| 01:20:07 | boyscout | CI: rubinius: 398d64d successful: 3024 files, 11753 examples, 35960 expectations, 0 failures, 0 errors |
| 01:20:27 | evan | maharg: reference to what? |
| 01:20:42 | maharg | how to use it and/or how it works |
| 01:21:06 | evan | no, not really |
| 01:21:12 | evan | we haven't had time to write much up. |
| 01:21:19 | evan | the code is the best source of that. |
| 01:21:24 | evan | and i can answer questions if you have them |
| 01:49:27 | boyscout | Implement String#partition. Fixies #165 - 98813f7 - Ivan Samsonov |
| 01:49:27 | boyscout | Cleanup the style and comment String#partition - b8cc7ee - Evan Phoenix |
| 01:50:23 | rue | I suppose the old MVM mechanism still applies in general. Some notable parts like Channel would be different |
| 01:52:09 | evan | nah |
| 01:52:14 | evan | Channel is entirely unrelated. |
| 01:52:34 | evan | we could wire it in via Channel if we wanted |
| 01:52:38 | evan | but thats not how it was before. |
| 01:52:57 | rue | No? Thought Channels were used for communication |
| 01:53:05 | evan | intra-VM |
| 01:53:08 | evan | not inter-VM |
| 01:53:27 | boyscout | CI: rubinius: b8cc7ee successful: 3024 files, 11758 examples, 35968 expectations, 0 failures, 0 errors |
| 01:53:27 | evan | inter-VM was handled via a simple shared mailbox |
| 01:59:51 | rue | Hmm |
| 02:06:21 | maharg | hm. Doesn't look like it actually works anymore? spawn_prim isn't hooked into anything |
| 02:07:10 | rue | Heh: http://www.wired.com/autopia/2010/01/nasa-puffin/ - I am not sure about the head-first approach |
| 02:08:02 | rue | Surely they could make it a J-shape cockpit and turn it in relation to the rotors |
| 02:10:59 | evan | maharg: yeah, it's completely busted |
| 02:11:05 | evan | it didn't make the C++ transition yet. |
| 02:14:22 | maharg | gotcha |
| 04:36:03 | evan | kronos_vano: you still around? |
| 04:36:14 | evan | when profiling, be sure to use -Xint right now |
| 07:36:44 | maharg | heh. Cute, if a little evil, how you have Object::symbol_p and the like testing bitfields on this, which may not actually be a valid pointer |
| 07:42:38 | evan | yeah, thats the same technique MRI uses. |
| 07:47:58 | maharg | oh yeah, I know the basic technique. But I mean the fact that you're doing it from in a C++ method to `this`. |
| 07:48:04 | evan | ah |
| 07:48:05 | evan | :) |
| 07:48:06 | evan | yeah. |
| 07:48:37 | evan | basically the methods on Object just have to be careful to use reference_p() and such to guard code that access klass and things |
| 07:48:43 | evan | it's turned out to be pretty easy to get right. |
| 08:14:04 | slava | yeah its a nice trick |
| 09:41:36 | maharg | it still uses a compacting collector, right? How does the VM code deal with moving Object* pointers between collects? Actually, even if it doesn't compact anymore, I can see that it still moves the pointer when promoting. |
| 09:42:02 | maharg | I see it storing Object*s in, for example, the lookuptables, but I don't see it doing anything special with them before doing so |
| 09:44:54 | slava | compaction relies on the gc knowing precisely which values are pointers |
| 09:45:27 | slava | I'm not familiar with the specifics of the rbx gc but typically you use a classical visitor pattern to identify pointers. |
| 09:46:22 | maharg | not really the angle I'm looking for. More an implementation detail of how rubinius deals with mutating pointers post-compaction. I understand the theory of GC in general :) |
| 09:46:54 | slava | sorry not sure what your level is :) |
| 09:47:18 | slava | I don't think it does compaction, and pointers that are promoted are updated in the mark phase |
| 09:47:24 | slava | along with pushing on the mark stack |
| 10:28:53 | khaase | morning |
| 10:29:21 | khaase | I have issues with rspec: http://gist.github.com/286729 |
| 10:29:48 | khaase | no, it is not an rspec issue |
| 10:29:55 | khaase | same happens with mspec |
| 10:31:13 | khaase | ok, it's my code |
| 10:31:17 | khaase | for some reason |
| 10:33:34 | kronos_vano | hm, common problem... I forget how to fix it... May be try to delete old *.rbc files. Or reinstall rspec... |
| 10:34:03 | dbussink | khaase: errors like that always point at stale rbc files |
| 10:34:22 | dbussink | khaase: probably some rbc that still refers verify_metaclass which has been removed long since |
| 10:34:41 | khaase | thought so, too |
| 10:34:50 | khaase | but i removed like anything |
| 10:35:02 | khaase | recompiled, installed all the stuff again. |
| 10:35:53 | khaase | i even removed all folders including anything, so the *.rbc files, too. |
| 10:37:28 | khaase | my fault |
| 10:48:09 | khaase | ok, next question: does rubinius use $RUBYOPT? |
| 10:48:16 | khaase | it seems to ignore it |
| 11:00:16 | dbussink | khaase: afaik it does |
| 11:00:26 | dbussink | khaase: it should pick up everything that is used in libraries etc. including this |
| 11:00:40 | dbussink | we've had reports of stuff breaking with complex rubyopt settings |
| 11:07:34 | khaase | i have a very simple RUBYOPT=rubygems |
| 11:07:53 | khaase | however, i still have to require "rubygems" |
| 15:47:25 | rue | maharg: ObjectFlags.Forwarded |
| 17:04:30 | evan | maharg: whats your question about compaction? |
| 17:23:42 | kronos_vano | evan, morning! -Xint What is it? |
| 17:23:51 | evan | it disables the JIT |
| 17:23:58 | evan | the JIT causes some strange profiling output right now |
| 17:24:01 | evan | i'm going to be working on that today |
| 17:24:02 | kronos_vano | ah, ok |
| 17:24:13 | evan | so for now, -Xint gets you the most accurate profiling output |
| 17:29:21 | kronos_vano | evan, Some methods writed in C++ (for example parsing number from string), some in ruby (Array#pack). Is there any criteria for that? |
| 17:29:36 | lypanov | dbussink: hey, around? |
| 17:29:44 | evan | kronos_vano: they should be in ruby unless they must be in C++. |
| 17:29:52 | evan | in the case of string parsing for a number |
| 17:30:10 | evan | we decided that it was of critical performance wise to make it C++ |
| 17:30:53 | kronos_vano | ok. |
| 18:47:31 | boyscout | Fix and expand the Module#include specs - 7091887 - Evan Phoenix |
| 18:47:31 | boyscout | Fix Module#include. Fixes #170. - 91b7863 - Evan Phoenix |
| 18:51:29 | boyscout | CI: rubinius: 91b7863 successful: 3024 files, 11761 examples, 35984 expectations, 0 failures, 0 errors |
| 19:10:15 | evan | zoinks |
| 19:10:21 | evan | there was no Module#include spec for cache effects. |
| 19:13:04 | boyscout | Add Module#include spec for cache effects - ea07e91 - Evan Phoenix |
| 19:13:04 | boyscout | Clear the method cache on Module#include - bb511f8 - Evan Phoenix |
| 19:17:00 | boyscout | CI: rubinius: bb511f8 successful: 3024 files, 11762 examples, 35986 expectations, 0 failures, 0 errors |
| 19:27:49 | rue | Ah, method cache effects |
| 19:47:42 | boyscout | Fix Module#module_function eval interactions - 60bb90e - Evan Phoenix |
| 19:51:42 | boyscout | CI: rubinius: 60bb90e successful: 3024 files, 11764 examples, 35988 expectations, 0 failures, 0 errors |
| 19:54:34 | evan | FUUUKC. |
| 19:54:58 | evan | wtf specs. |
| 19:58:37 | evan | ug. |
| 19:58:47 | evan | these autoload specs have some fucked up errors. |
| 19:58:51 | evan | that mask what's going on. |
| 20:05:40 | boyscout | Fix spec bug in autoload. Autoload uses require internally. - 2275c01 - Evan Phoenix |
| 20:05:40 | boyscout | Remove Module#autoload tags - b730d3f - Evan Phoenix |
| 20:21:08 | boyscout | CI: Commit b730d3f failed. http://github.com/evanphx/rubinius/commit/b730d3ffb38117077ffdd4f03b5d375cbba28f0d |
| 21:16:58 | boyscout | Add missing file - 8eaeb08 - Evan Phoenix |
| 21:20:57 | boyscout | CI: rubinius: 8eaeb08 successful: 3024 files, 11766 examples, 35992 expectations, 0 failures, 0 errors |
| 21:25:29 | evan | danku boyscout |
| 22:23:19 | rue | Man, I admire his effort level, but I am going to have to start filtering out Pack's -core messages |
| 23:42:10 | evan | huh. |
| 23:42:20 | evan | Bignum doesn't have it's own < method |
| 23:42:22 | evan | it uses comparable. |
| 23:42:23 | evan | weird. |
| 23:44:31 | BrianRice-work | what does comparable's default method do? |
| 23:44:36 | evan | call <=> |
| 23:44:49 | evan | and return true if <=> returned 0 |
| 23:45:29 | evan | it's not unreasonable |
| 23:45:44 | evan | just an extra level of indirection I didn't expect on something as performance critical as Bignum |
| 23:45:51 | BrianRice-work | oh, I see, the base method is <=> which is sensible. maybe not the best performance but not bad |
| 23:45:57 | evan | righto |
| 23:46:06 | BrianRice-work | well, how often does bignum performance come up? |
| 23:46:54 | BrianRice-work | we noticed that our 64-bit images don't need bignums in any normal code |
| 23:49:07 | evan | yeah |
| 23:49:11 | evan | not as often as they used to |
| 23:49:20 | evan | thats why i suspect it delegates to Comparable. |
| 23:49:26 | BrianRice-work | ok |