Index

Show enters and exits. Hide enters and exits.

01:11:31dwaitesigh
01:12:47dwaiteI think brixen has ragel rage
01:13:09brixenwhat's that?
01:13:16brixendo I need to see a doctor?
01:16:06dwaitemaybe a therapist
01:16:18dwaiteor if you don't get to move away from pack/unpack soon, a priest
01:16:52dwaiteI was trying to send you an IM earlier asking if you had put your slides up
01:17:05dwaitebut apparently google broke google talk for a good % of external users :-/
01:17:10brixengchat probably doesn't like you
01:17:19brixenit didn't like me week before last
01:17:23brixensharing the wealth and all
01:17:38brixenI'll put the slides up in a bit
01:17:47dwaitewhen you say gchat, do you not mean the gtk irc client?
01:18:02dwaiteit appears to be a race condition in the jabber dialback protocol
01:18:13dwaitewhich is basically a distributed asyncronous state machine
01:18:23brixenoh, I mean running google talk from inside the email browser window
01:19:16dwaitebut if you tried to write it out that way, you'd either go mad or start to be able to visualize in a fourth perpendicular dimension
01:20:58dwaiteshakes his fist at jer for not only creating dialback, but being able to comprehend it
01:21:49dwaitebut basically it means - you can probably send me a message right now, but I cannot respond, including sending you presence to say I'm online
01:23:22boyscoutA bunch of cleanup - 83feb62 - Evan Phoenix
01:24:48brixendwaite: good thing there is IRC :)
01:55:51evan:/
01:56:00evanso, our benchmarking results have been inflated.
01:56:18evanbecause Process.times on MRI is NOT wallclock at all.
01:56:20evanand ours was.
02:00:54jakedouglasremind me to discuss some profiler strangeness with you when my brain recovers
02:00:58brixeninflated == we are slower or faster than we though?
02:01:02brixenthought?
02:02:07evanwe are faster than we thought.
02:02:53evanoff for a jog, will look into it a bit more
02:55:30boyscoutStyle fix of benchmark.rb - 9689249 - Evan Phoenix
02:55:30boyscoutUse getrusage for Process.times - 27928fe - Evan Phoenix
03:37:51Asher-i have put together a c extension that relies on the call stack - to what extent will rubinius support such calls out of the box?
03:38:04Asher-and is there an appropriate way to proceed in figuring out how to adapt the extension (if necessary) to work?
03:44:23brixenAsher-: your ext will not work with rbx if it depends on the MRI stack frames
03:44:36brixenwhet does you ext do? and have you tried compiling it with rbx?
03:47:21Asher-i haven't tried yet - does rbx have ruby access to its own internal stack?
03:47:28Asher-i figured it wouldn't work so i hadn't tried
03:47:57Asher-it does an object oriented backtrace and gets sender/caller
03:49:38Asher-i would like to support rbx - happy to write whatever additional code needs to be done
03:49:49Asher-just not sure where to look - weren't any clear docs differentiating those internals that i saw
03:49:52Asher-although i imagine they exist somewhere?
03:50:09brixenwe already have an object oriented backtrace
03:50:27brixenkernel/common/backtrace.rb :)
03:50:28Asher-is there a specific place documenting what rbx has that mri does not?
03:50:34Asher-b/c mri does not have one :)
03:50:40brixenno docs really, just the code
03:50:47Asher-ok
03:50:47brixenyou should clone rbx and check it out
03:51:04Asher-yeah seems the next step
03:52:04jakedouglaswhat's the difference between a CompiledMethod and a VMMethod? (why are there two things to represent a method?)
03:52:31brixena CM is an abstract set of opcodes, a VMM is something the VM actually runs
03:54:12jakedouglaswhat is 'abstract' about it?
03:54:58brixenthe VM doesn't actually execute it :)
03:55:14brixentake a look at how a VMMethod is created
03:55:27jakedouglasi can tell
03:55:50jakedouglasbut if you create a vmm from a cm, the point of the cm is just to be a bytecode container or what?
03:56:02jakedouglaswhy have both? just organizational?
03:56:23brixenthe vmm has specializations for the platform
03:56:52brixenCMs are completely portable
03:57:06jakedouglasoh
03:58:41boyscoutSpecs for String#unpack IiNV. - 769a7fc - Brian Ford
03:58:41boyscoutSpecs for String#unpack LlQq. - a98c0fa - Brian Ford
03:58:41boyscoutString#unpack IiLlNVQq. - d7d4469 - Brian Ford
03:58:41boyscoutRemove removed files from load_order. - 9a9062d - Brian Ford
03:59:32brixenhm, CI seems hung and I'm soo hungry
03:59:43brixenI think I'm gonna grab some food and check back in a bit
04:00:04jakedouglas!!!!
04:01:17brixenjakedouglas: btw, better docs coming RSN
04:01:23jakedouglasniiiiiice
04:01:36brixen1.1 is supposed to be all dev tools, and docs are an essential part of that
04:01:39jakedouglasall this crazy vm shit im learning at jvm summit has me digging around trying to figure everything out
04:01:54brixenI want to get a rbx doc command running that starts a local web server of browsable docs
04:02:24brixenok, bbiab...
04:02:27jakedouglask.
04:09:26dwaiteI didn't really get the java io vs nio thing earlier today that was flying around
04:09:34dwaitewasn't it basically saying that the java nio implementation sucks?
04:12:51dwaitehe also seemed to say that java's minimum per-thread size was 48K, but doesn't each thread get its own copy collector?
04:14:35dwaiteahh in 6 it looks like it is one per cpu
04:18:24brixenwhat we need are a cannonical set of apps that demonstrates each of these strategies in optimal context
04:21:26brixenso we have more to analyze than the sound bytes into which these discussions normally devolve
04:21:44brixenand it would serve to highlight assumptions
04:22:03brixeneg http://twitter.com/cscotta/status/19699128358
04:22:11brixenand http://twitter.com/cscotta/status/19702967623
04:22:54brixenthe water may be sweet and warm, but it's hardly ever so idyllic you want to jump in without sticking a toe in first
04:25:25Asher-this is crazy… a channel where like… people are actually working on stuff?
04:25:35Asher-puts #ruby and #ruby-lang and #rubyonrails to shame :P
04:26:34jakedouglasdwaite: just forget about it, its a waste of time :)
04:26:50jakedouglasthere are cooler things to care about
04:28:35dwaitejakedouglas: very true
04:29:51dwaitebrixen: really a library that implements user-mode threading is really implementing async I/O and context switches, its not even like it is undocumented API :)
04:30:51dwaiteso if you think you can handle something like OS thread affinity or eliminate stacks or what not, thats when you should start implementing your own I/O system :)
04:31:22dwaiteI did the fiber to make evented IO systems look like synchronous systems thing last year, and in the end what I had really done was
04:31:31dwaiteimplement green threads inside ruby to get around the GIL
04:32:17dwaitebut thats not to say that there couldn't be say memory efficiencies gained from doing that
04:32:34dwaitehave ruby fibers use a spaghetti stack ;-)
04:32:52dwaite(since they can fall back on the calling thread's C stack)
04:33:30jakedouglasif you're talking about doing it in ruby then you also avoided (some) inevitable disaster of trying to use everyones gems with threads, which they certainly have not been designed for
04:33:31dwaitecooperative threading models also are way easier to comprehend, and its nice not to have to think about threads
04:34:12jakedouglasit doesnt really have anything to do with a gil
04:34:37jakedouglasMRI anyway does io-oriented thread scheduling
04:34:47slavanot implementing pre-emptive threads from the start is my biggest regret with factor
04:34:52jakedouglasso you're getting the same thing as fibers really. fibers are just more simple.
04:34:53dwaite*nod* well threads are not exactly the peak of excellence anyway. I've had to debug way too many threading problems in C++ code
04:35:00slavait sounds like rbx is going to ditch the GIL pretty soon which is really good news
04:35:11dwaiteit actually doesn't even matter if the problem is related to threading or locking or anything
04:35:24dwaitejust every problem becomes 10x harder to cope with when you are trying to debug it under real threads
04:35:38jakedouglaswell
04:35:51jakedouglasthere are a lot more situations where you can reason about cooperatively scheduled execution
04:36:27jakedouglasnot that it can't still be confusing sometimes
04:36:49dwaiteyeah the ability to run threaded is nice
04:37:00dwaiteits just one I wish I could find more ways around ;-)
04:37:17dwaiteand the worst bugs are kernel drivers
04:37:45dwaiteI was writing support for a video card and accidently added a floating point operation without thinking about it :)
04:38:14dwaiteall of a sudden random applications had a corrupted FPU stack/registers
04:38:38DefilerThey probably weren't using them optimally anyway; they probably deserved it.
04:39:38dwaiteit didn't crash any apps
04:39:46dwaiteit was more like their assumptions about space and time were challenged
04:41:37dwaitelooks like the jabber issue with google talk is more busted on their end than I thought
04:41:43dwaiteits not something that I think I can work around with software :)
04:42:45dwaite(thank goodness, I was already starting to learn how to read erlang, last thing I want is to know how to write it too)
04:44:35jakedouglasyay, im finally figuring out how this rubinius thing works
04:46:46jakedouglasseems so much easier after listening to everyone talk about forkjoin and fence instructions all day
04:50:19brixenjakedouglas: envious! :P
04:50:52jakedouglasits been pretty hard keeping up but i have without a doubt learned A LOT
04:51:05jakedouglasits been kind of shitty at times but i'm glad i came
04:51:18slavaI think I changed more lines of code today than in the last two weeks
04:52:23slavatime for a blog post!
04:55:50brixenslava: what have you been working on?
04:56:16slavabrixen: I overhauled FFI callbacks. the body of the callback used to be compiled with the base compiler, and now its compiled with the optimizing compiler
04:56:33brixenah cool
04:56:33slavabrixen: so there's a theoretical performance improvement, but the main benefit is the static checking done on the body
04:56:45brixenI saw your tweet on that
04:56:58slavabrixen: previously if your callback took two parameters but the code popped three values from the stack you'd get a runtime error
04:57:27slavawhich was really annoying because the default error handler at the top level of a callback kills the VM (there's no unwinding or throwing past C frames)
04:57:43brixenah yeah
04:57:49brixensounds like a good post!
04:58:07slavaI'm going to do a write up of some other recent FFI changes which were more interesting
04:58:34slavaFFI calls are broken down into several IR instructions now, and insetad of calling out to the VM to box and unbox arguments they share the rest of the codegen's boxing support
04:58:50brixenvery cool
04:59:02slavaso you can compute some floats and then call a C function inside a loop and you won't get any allocation, or at least less than before
04:59:08dwaitenice!
05:00:14dwaitehaha
05:00:15dwaiteok
05:00:23dwaiteI think I know how to 'fix' google chat
05:02:27slavabrixen: ffi calls now compile without saving/restoring all live values to the operand stack
05:02:42slavabrixen: instead the register allocator spills live registers and records a bitmap of which spill slots contain GC pointers
05:02:55slavabrixen: the next step is to do this for all calls
05:03:36brixenawesome
05:03:57slavathat was kind of my plan all along, get it working for ffi calls first since its a bit simpler
05:05:01slavaits weird how over time the stack went from being an integral part of the implementation to just syntax that the compiler eliminates
05:05:11slavamaybe its just a bad idea overall and I should've gone with a mainstream syntax
05:05:43brixenprobably :)
05:06:01brixenI'm still planning on targeting your stack with my Poison syntax :)
05:06:04slavabut you guys have a funny setup too. ruby syntax with named variables -> stack bytecode -> LLVM IR with named registers
05:06:52slavamaybe the forth people have poisoned all VM implementors with their legacy 70's ideas
05:06:57brixenwell, I spent a while studying the Jimple papers
05:07:13evana busy evening I see.
05:07:15brixenI'm still curious about trying a register backend to the compiler
05:08:06brixenevan: could you please kick CI
05:08:18evanyep
05:08:24evanthe agent is running on it too
05:08:28evanso I can see where it's hung.
05:08:34brixenyeah, that's nice
05:09:08evanhm, not connecting.
05:09:13evancan't find the tmp file.
05:09:14evandang.
05:09:16evanoh well.
05:09:20evangdb, here I come!
05:09:23dwaite:)
05:09:23brixenheh
05:09:31brixenthis is interesting: http://skitch.com/timfelgentreff/dqcub/microsoft-excel-book1
05:09:40dwaitebrixen, you get my IMs? you are making me feel nervous that I didn't actually fix the prob ;-)
05:10:04slavabrixen: regarding jimple, I'm not sure if constructing SSA in javac is worth it
05:10:13slavaits probably better to do type feedback-based inlining first
05:10:25slavathen build SSA and do fancy optimizations at runtime
05:10:49brixenslava: oh, kinda like evan is already doing? :)
05:10:56slavayeah
05:11:13slavayou could still have a register-based bytecode, its just not worth computing phi function placement if you'll have to redo it all after inlining anyway
05:11:44evanslava: i think of all our data forms for the execution as being tailored to their duty
05:11:52evanstack bytecode is fast and easy to interprete
05:12:06slavaits also easy to generate
05:12:12evanand it's got certain restrictions that allow it to be converted into named values within LLVM SSA easily
05:12:16evanbest of both worlds.
05:12:44brixenthe only thing missing is an academic paper on the subject :)
05:12:53evanvirtual registers would have been easier converting to LLVM SAA
05:12:54evanSSA
05:12:57evani won't deny that.
05:13:06evansince I could do it blindly.
05:13:36slavayou guys should write a paper for DLS or VMIL
05:13:47slavathere's still time for VMIL '10, deadline is aug 9th
05:14:14slavaor there's always DLS '11
05:14:33evanI think I'd prefer to aim for next year
05:14:44slavaby then you'll be state of the art :)
05:14:50evan:)
05:14:57evana few more bowling pins to knock down
05:15:00evanspeaking of which
05:15:10evantonight i'm starting a hydra branch
05:15:17evanwhich removes the GIL.
05:15:18evan:)
05:15:28brixenwoot!
05:15:38jakedouglaswhere are the places that need finer locks?
05:15:51evanjakedouglas: thats what we're going to find out
05:15:52evan:D
05:15:54brixenjakedouglas: if we had all that, it be pretty easy
05:15:55jakedouglasheh.
05:15:56brixenheh
05:15:59jakedouglas:)
05:16:06evanI've got a few ideas
05:16:12evanand i'm going to go overboard with locks at first
05:16:20evanand then scale them back
05:16:49evanfor instance, a lock in capi
05:16:57evanto keep only one native method running at a time
05:17:13evansince extensions are not accustom to running with true concurrency
05:17:18evanthey might be fine, but I don't know
05:17:24evanso I'll put in a lock in for now.
05:17:58jakedouglasdamn. im totally understanding this vm stuff way better now. i dont quite have the big picture down intuitively but i can follow most of the code except the llvm stuff.
05:18:06evanyay!
05:18:11brixensweet
05:18:19evanI need to keep refactoring LLVM a bit
05:18:21jakedouglasjvm langs whipped me into shape heh
05:18:31slavajakedouglas: did you attend last year?
05:18:31evanit's still shotgunning data around
05:18:37jakedouglasslava: no.
05:18:55slavajakedouglas: I got an email about it but didn't go this year because I've been way too busy
05:19:15jakedouglasi've tried reading some llvm tutorials but i find it god awful confusing
05:19:32slavaits a portable assembly language
05:19:47slavalook at the textual form of LLVM IR before you dive into the API
05:19:54jakedouglasoh. is it easier?
05:19:55evanjakedouglas: at the level we use it, it's simply an AST
05:19:59slavaand play around with the command-line LLVM IR compiler. it will make sense immediately
05:20:03evanyou build a tree of what you want to execute
05:20:14evanso
05:20:19evanthe way I learned llvm was using llvm-gcc
05:20:21evanwrite a C program
05:20:29jakedouglascompile to llvm ir?
05:20:36evanthen use llvm-gcc -S to see the llvm ir in LL form
05:20:40evanLL is the text format
05:20:42jakedouglasi see.
05:20:57evanthats how I built up how to use LLVM
05:21:05evani'd say "ok, i want a condition and then a call"
05:21:10evanso i'd do that in C, check out the LLVM IR
05:21:15jakedouglasright
05:21:19evanthen you can do llc -arch=cpp
05:21:32evanto even have LLVM convert the IR to a C++ file that would reconstruct that IR
05:21:37evanso you can see how the C++ API works
05:22:04dwaiteevan: they have a documentation directory now
05:22:15evanwe use that to bootstrap part of our JIT actually
05:22:27jakedouglasi dont really understand what you mean by that you use it as an AST. i dont have an concrete understanding of what an AST really is though
05:22:45evanjakedouglas: just a tree of objects
05:22:50evanthats all.
05:22:56jakedouglashmm.
05:22:58evaneach object is an operation
05:23:15dwaitejakedouglas: compilers are about shifting the essence of the commands in your program around into different forms until it finally gets to a point where the CPU inside your machine understands it :)
05:23:17evanyou can check out the AST for rubinius using -A
05:23:20evanbin/rbx compile -A <file>
05:23:29jakedouglasok.
05:23:34brixenyes, quite helpful
05:23:47evanthats not the AST that LLVM uses, but it's helpful for understanding what an AST does.
05:23:52dwaiteAST is kinda the first form; elements of the syntax represented as objects, where expressions and subexpressions wind up forming a tree
05:24:24dwaiteat least thats how i've always understood it
05:25:05slavaLLVM IR differs from a typical AST in that it is flat
05:25:12slavayou don't say (x+y)*4
05:25:20brixenjakedouglas: there are some really helpful books here: http://www.engineyard.com/blog/2009/rubinius-the-book-tour/
05:25:21slavainstead its more like z = x+y; w = z*4;
05:25:22jakedouglasIR is more similar to machine code, ya?
05:25:25boyscoutCI: Commit 83feb62 failed. http://github.com/evanphx/rubinius/commit/83feb6254c474b29e7623985589dffab4a827557
05:25:30brixenjakedouglas: in particular, engineering a compiler
05:25:31evanslava: well, yes and no
05:25:40jakedouglasbrixen: tanks
05:25:42jakedouglasthanks
05:25:43evanslava: at the AST level, you do have (x+y)*4
05:25:47slavaoh
05:25:52brixenjakedouglas: IR is any intermediate representation
05:26:00evanthe SSA form you see is it linearized and broken out for the text representation
05:26:03brixenjakedouglas: that book (EAC) has a great section on IRs
05:26:10evanbecause there is really no way to textualize an AST
05:26:32evanbrixen: erm, fixing CI.
05:26:46jakedouglasyea the output of compile -A makes plenty of sense.
05:26:49brixenevan: I already pushed a fix
05:26:56evanok, thanks
05:26:58evani'll kick it again.
05:26:58brixenn/p
05:27:18evanjakedouglas: thats an AST
05:27:21evanpretty easy.
05:27:36evanslava: LLVM's SSA is oriented toward that text output because you can give each node a name
05:27:36jakedouglasright
05:27:41evanand than name is what's used in the text output
05:27:49evanbut if they have no names, they get names like %3
05:27:59evanie, just an incremented number within that function
05:28:28slavaI see
05:45:36evansweet
05:46:06evana little C++ class hierarchy + RIAA + macros means I can add locking to a C++ class very easily
05:46:42evaninherit from thread::Lockable
05:46:52evanput LOCK_ME in any method you want to lock
05:53:42brixennice
05:58:12evanbrixen: i'm also going to add a lock_object and unlock_object instructions
05:58:22evanand Object::lock()/unlock() in C++
05:58:28evanthat use a mutex in the inflated header
05:59:18evanwe can get fancy with thin locks once it's working
05:59:59slavaevery object will have a lock?
06:00:17evanfor this first experiment, yes.
06:00:32slavaany thoughts on whether or not arrays and hashes should be thread-safe?
06:00:39evanprobably not
06:00:45evancharlie and I have discussed that a bunch
06:00:50evanin jruby they're not.
06:01:00slavaat least mutating an array from multiple threads should be memory-safe though right?
06:01:05evanyes
06:01:10evanmemory-safe is aim 1
06:01:18slavabut something like pushing concurrently might overwrite the second element with the first
06:01:22slavabecause if you do that you suck
06:01:25slavabut if you suck your VM shouldn't segfault
06:01:27evanwhich must be true even for misbehaving ruby programs
06:01:49evanyes
06:01:53evanruby code can't segfault the VM.
06:01:54evanas a rule.
06:02:20slavaperhaps factor should have something like D's safe/trusted/unsafe qualifiers
06:02:30slavasafe code can only call safe and trusted code, trusted code can call unsafe code
06:02:44evanwell, FFI is a bit like that
06:02:51evansinec it's unsafe.
06:03:12slavabut you don't enforce this kind of discipline right?
06:03:20evanno.
06:03:21slavaso yeah, any FFI call would make your method unsafe
06:03:30evanwell, "yes" if i say that all ruby code is trusted code
06:03:31evan:D
06:03:33slavaand you'd have to wrap it in a trusted method to check stuff etc
06:03:39slavatrust the ruby
06:04:06slavaso there's this FFI issue that factor has, I'm not sure what a good solution is
06:04:19slavaif you have a function declared to take a string, you can pass a string object or false
06:04:26slavaif you pass false, its unboxed to a null pointer
06:04:29slavawhich can segfault
06:04:35evanyep.
06:04:44evanbut it's the thing you called that segfaults, yes?
06:05:01slavayes, but the issue is that usually if you pass a boolean where a string is expected, you'd see some kind of type error or something
06:05:13slavabut here the value can bubble down 10 layers and end up in an FFI call
06:05:19slavaeven though you called some nice high level API
06:06:19evanwell
06:06:25evanyou have to be able to pass NULL some way
06:06:32evanbecause thats a normal C API duty
06:06:49slavaI was thinking of changing it so that in this case you have to declare the parameter as a pointer, and not a string
06:07:00slavaand then if yo uwant to pass a string or null, convert the string to a byte array manually
06:07:15slavabut yeah, FFI raises all kinds of safety issues
06:07:30slavathere's no good solution. VM extensions are no better, they just force you to write boilerplate and think about everything
06:07:36slavaits harder to fuck up
06:07:46evanenforcing a form of type safety by not allowing null is possible
06:07:58evanbut you're just making your users go through hoops to call a C function that could crash anyway
06:08:17slavayeha
06:08:23slavaclearly we need to get rid of C altogether :)
06:08:28evan:)
06:08:48evandevising a simple language that allows you to write VM extensions is an option
06:10:24brixenback
06:10:30evanwelcome back b.
06:10:34brixenwhat's with a bar closing at 11pm
06:10:38brixenlosers :)
06:10:41evanin portland?!
06:10:43slavabrixen: while you were gone me and evan decided to eliminate C
06:10:44evantell the cops
06:10:46brixenheh, yeah
06:10:54brixenslava: good plan
06:11:00slavabrixen: littledan tells me that in irvine everything closes at 10pm because they're a bunch of suburban pansies
06:11:03brixenslava: can we all use Go now?
06:11:03evanbrixen: that must be a union violation in portland.
06:11:13evanslava: that is totally true of irvine
06:11:17evanit's mega-burbia.
06:11:32littledanbut they don't have bars in Irvine in the first place
06:11:42littledanyou have to go to one of the beaches or santa ana
06:12:14evanhey littledan is in here!
06:12:20littledanyeah!
06:12:24evanwelcome!
06:12:29slavalittledan is the famous tracing JIT researcher at UC Irvine
06:12:32evanyou should come hang up in LA sometime
06:12:38evanoh man
06:12:39evanwe must meet
06:12:41littledanI'd love to
06:12:42evani'd love to talk JITs
06:12:47evanI never get to.
06:12:53littledanyou live in LA?
06:12:56evanyep!
06:12:57evanHollywood.
06:13:41evanhey! bin/rbx -v booted without the GIL!
06:13:46littledanwell I'm free this weekend
06:13:50evanI wonder if this will be like when we ran rails
06:14:01evanwe thought it would be hard, I hacked for a few hours, and it worked.
06:14:06evanthat would be hilarious.
06:14:18evanlittledan: we weekend is looking full, going to the X games on saturday
06:14:28evaner. s/we/my/
06:14:58slavabut how many threads are actually running?
06:15:03evan1!
06:15:08evan:)
06:16:08evanlittledan: you free at all during the week? or are you hard at working at UC Irvine
06:16:09brixenheh
06:16:21slavaevan: s/hard at/hardly/
06:16:32evanCOWORKER ZING ALERT
06:17:08brixenevan: hurry, run some benchmarks and write a blog post with a very tiny disclaimer at the end :)
06:17:16evanhah
06:18:22littledanevan, I can take weekdays off and work on the weekend in academia
06:18:50brixenwho works in academia?
06:18:55evanwe should definitely work something out.
06:19:11littledandoes
06:19:14evanlittledan: maybe somewhere between here and there?
06:19:21littledanwhy?
06:19:28littledanI'd rather go there
06:19:32evanha!
06:19:33evanok
06:19:35evaneasy for me!
06:20:03slavaI'm just sitting on the sidelines, watching this relationship blossom.
06:20:17evanwe can talk shit about slava
06:20:18evanerr..
06:20:20evani mean...
06:20:21evanum...
06:20:25evantalk about how awesome he is.
06:22:04brixenyou guys can discuss slava's latent syntax envy :)
06:22:12evanhah
06:38:50evanslava: do you use FFI for the UI
06:39:04slavayes
06:39:15slavathat was the original motivation, along with IO
06:39:48evanso you've got enough of the platform APIs mapped to open windows and draw I guess?
06:40:08slavabasically yeah
06:40:40brixenslava: we should collaborate, rbx should have a killer GUI app
06:41:51brixenslava: also ☃
06:41:54slavaone day I'll start doing real-world shit
06:42:10brixenhttp://railssnowman.info/ :)
06:42:43brixenI really hope they rename that to _ewww and not just _e
07:13:01evanok, off to bed.
07:13:07evanmore fun with hydra tomorrow.
16:19:19brixenahh, the smell of fresh spec failures in the crisp overcast morning
16:19:26jakedouglas:p
16:20:09jakedouglasyou're making me miss the NW.
16:20:51brixenheh
16:20:58brixenI love sun and warm
16:21:04evan:D
16:21:05brixenno idea what I'm doing living in the NW
16:21:07jakedouglasi guess ill be home in about 13 hours though.
16:21:19brixenjakedouglas: what's on the schedule today?
16:21:52jakedouglashmm. im sitting at the hotel waiting for them to take me over, slept in a bit today. lets see...
16:21:58jakedouglasoh, they are actually giving talks on languages today!
16:22:02jakedouglasamazing
16:22:02brixenslacker!
16:22:17jakedouglasyea i feel like im getting sick or something :/
16:22:22brixen:(
16:22:33brixensounds like a bunch of folks are getting summer colds
16:22:36jakedouglasi'm currently missing groovy
16:22:36brixenteh suck
16:23:06jakedouglasthen php, kawa, workshop with rich hickey, someone from oracle talking about something, scala, fortress
16:23:43brixenjakedouglas: you said these are being recorded, right?
16:23:56jakedouglasyea.
16:24:00brixenexcellent
16:24:47jakedouglasoh yea. workshop with cliff click is today too. that will be interesting
16:25:03brixenyeah
16:25:46jakedouglashes going to talk about azul hardware stuff and/or jit details
16:26:03evanjakedouglas: quick! transport me in!
16:26:11evani wanna hear cliff talk about that stuff!
16:26:38brixenjakedouglas: do you have an iphone 4? you could facetime with evan live from the talk
16:26:44jakedouglasheh
16:26:48evanYES
16:26:53evanYYYYYEEEEESSSSS
16:26:57evan:)
16:26:57jakedouglasi would be happy to broadcast with skype or something
16:27:09evanis Guy Steele there?
16:27:12jakedouglasthe workshops are not recorded so…yea you would miss out on that one
16:27:14jakedouglasi don't know.
16:27:21evanhe's the main fortress guy
16:27:22evanor was.
16:27:32evanbefore the Snoracle shakeup
16:28:03jakedouglasah. i don't think so. christine flood is whos speaking about fortress today, she's been nice enough to give me rides every day too :)
16:28:38evanah cool.
16:28:50evani wonder if fortress is still using UTF-8 operators
16:29:00jakedouglasheh
16:30:15jakedouglaswell, maybe i can do an incognito skype session later if you really want to hear click
16:31:34evanhehe
16:31:37evannah, it's fine.
16:31:46jakedouglasok :)
16:31:53brixenjakedouglas: take really good notes!
16:31:59jakedouglasheh ok.
16:32:04brixenvebatim
16:32:07brixener ver
16:32:32brixenor HD recorder, whatever is easiest :)
16:33:17jakedouglaswell i guess it depends on how he does it
16:33:32jakedouglasi think yesterday what was going to be a 'mini cliff workshop' ended up being kind of a talk, and i think that got recorded
16:33:59jakedouglasso there may be something for you to watch later.
17:18:35evanso, happily for me
17:18:45evani wrote a bunch of code hydra needed when I was working on the JIT thread
17:18:55evannamely, the ability to request all other threads stop
17:24:05brixenyeah, seems like you've been planning for this day for some time :)
17:24:15evanrad, concurrent allocation and full stop GC seem to be working
17:24:20brixenwoot!
17:24:50evanthe slab work I did just a few weeks ago was critical.
17:25:07brixenyeah
17:25:50jakedouglaswhat did i miss?
17:26:03brixenjakedouglas: it's all working
17:26:26jakedouglaswhat is all working
17:26:43brixenheh, evan's experiments with concurrent allocation and full stop GC
17:26:49evan:)
17:26:52evanjakedouglas: removing the GIL.
17:27:04jakedouglaswhat is 'full stop gc'? you mean you're getting other threads to stop for your gc to run?
17:27:15brixena very long time ago, before we ran rails the first time, rue would ask "what did I miss" and I would say, "we run rails now"
17:27:25jakedouglasheh.
17:28:01brixenjakedouglas: yeah, stop-the-world GC requires all the threads mutating objects to pause
17:28:39jakedouglasuh huh
17:29:45brixenman, I hate C printf
17:29:50brixensince when is %c type int
17:29:53brixeneffen eff
17:58:24evanrad, it's working.
17:58:44goyox86evan: woot! :]
18:00:38brixenvery cool
18:00:43goyox86evan: what does exactly means "concurrent allocation"?
18:01:08evangoyox86: means 2 or more threads running in true parallel allocating new objects
18:01:35evanand one deciding "I need to GC" and getting cooperation from all threads in the system to suspend so GC can happen
18:03:55goyox86evan: each vm run on a OS thread? , and when you say "2 or more threads", you are talking about green ones?, correct me if i'm wrong
18:04:32evanno
18:04:34brixengoyox86: native threads, true concurrency
18:04:36evanrubinius doesn't use green threads
18:04:40evannative threads
18:04:45evanhydra is the branch i'm working on now
18:04:47evanthat removes the GIL
18:04:53evanallowing for true concurrency
18:06:28goyox86evan: :] but shotgun used green threads isn't? i remember some adam gardiner post, talking about that, or again i'm wrong :]
18:07:14goyox86evan: an very old post i think
18:07:18goyox86a*
18:07:25evanreally old
18:07:26evanreally really old.
18:07:29evan2+ years.
18:07:41evanshotgun did use green threads
18:07:45evanas did the really early C++ stuff.
18:09:32goyox86evan: how do you keep track of all the threads mutating objects?, you just stop that ones? or again i'm wrong :]
18:10:07evani don't follow
18:10:10evani'm not sure what you're asking.
18:11:05goyox86evan: a few minutes ago brixen said -> jakedouglas: yeah, stop-the-world GC requires all the threads mutating objects to pause
18:11:24evanyep.
18:13:37goyox86evan: well bro, keep working on hydra :), remove taht fu***ing GIL
18:13:43goyox86fades
18:14:01evan:)
18:14:10evansorry, to answer your question
18:14:13blowmagedid someone say something about removing the gil?
18:14:13evanthreads checkpoint
18:14:17blowmageperks up
18:14:21brixenimagines evan spraying liberally with his can of GIL-be-gone
18:14:22evanblowmage: yep, working on it now.
18:14:29blowmageniiiiice!
18:15:14evangoyox86: one thread requests all other threads to stop, and then waits for them to check in
18:15:31evanother threads check once in a while if they should stop
18:15:37blowmageevan: will hydra be available on your github repo?
18:15:42evanyep
18:15:44evanas a branch
18:15:58evani'm getting the basics ironned out
18:16:01evanthen i'll push it.
18:16:11evanit will probably still deadlock and crash sometimes
18:16:24goyox86evan: that was i was asking for :)
18:16:38evangoyox86: :) there ya go!
18:32:24postmodernout of curiosity
18:32:32postmodernhow is === implemented vs. kind_of?
18:32:52evanthey're the same
18:33:17evanModule#===(obj) can just be
18:33:20evan obj.kind_of? self
18:45:17postmodernnice
19:06:14boyscoutFix for the following Marshal.dump specs: - 15ee649 - Jose Narvaez
19:06:14boyscoutRemove tags for passing Marshal.dump specs - 0fe55af - Jose Narvaez
19:06:40dbussinkgoyox86: you should ask evan for commit access :)
19:08:42goyox86evan: can i have a commit bit ;)?
19:08:50evansure
19:09:23evanok, added
19:09:25evanbe good
19:09:25evan!
19:10:36goyox86is happy, joined the "Entire World Domination (for ruby)" team :]
19:10:44evan:D
19:11:03tarcieriso GIL free branch you say?
19:11:05tarcieri:)
19:11:22evan:D
19:11:29evantarcieri: so far, working pretty well.
19:11:45tarcierischwee
19:11:49tarcierimoving to fine grained locks or what?
19:11:50evanconcurrent allocations and stop-the-world working well.
19:12:04tarcieridoes your GC run in its own thread?
19:12:05evanyep.
19:13:01evanfine grain locks yes,
19:13:03evangc thread, no.
19:13:11tarcierio
19:13:15evanwhichever thread wants to GC does the GC
19:13:19evanit asks all other threads to stop
19:13:24tarcieriok
19:13:26evanand waits
19:13:43tarcieriwell sounds dope
19:14:15evanthis might end up being like when we first ran rails
19:14:23evanI was pessimistic about it for a long time
19:14:30evanbut just decided to try one day
19:14:35evanabout 24 hours later, it ran.
19:14:43jakedouglasyou say 'concurrent allocation', do you allocate from something thread-local? or do you have some fine lock in it
19:14:52evanjakedouglas: thread-local
19:15:05evaneach thread gets a slab of memory to access without a lock
19:15:11evanand only locks to grab a new slab.
19:16:11jakedouglasdont you have to add the new obj to some central thing?
19:16:20evanno
19:16:26evanlike what?
19:16:29goyox86which are the current GIL free Ruby implementations, JRuby?
19:16:30jakedouglasoh. it's already part of the central thing?
19:16:40evanwhat "central thing?"
19:16:45evangoyox86: macruby too
19:17:45goyox86evan: what do you know about maglev?, it's not GIL free?
19:17:53evanthat I don't know.
19:18:03jakedouglasi duno, does the gc only look at objects for the thread its running on? i feel that i have something conceptually wrong. maybe you could give a summary :p
19:18:03dbussinkjakedouglas: there is no central thing, just that the gc needs to know which slabs are used, that's why asking for a new slab is locked
19:18:11tarcierievan: it'd be a pretty interesting state of affairs if rbx and jruby supported concurrent multithreading and YARV didn't
19:18:38evantarcieri: well, seems like thats going to happen.
19:18:40dbussinkevan: btw, did headius succeed in triggering you a bit with his jruby c ext mongrel benchmarks? ;)
19:18:49evanno.
19:18:56evanwhich benchmarks?
19:19:24evanjakedouglas: to allocate an object, we setup the object header in memory with the class and clear the body, thats it.
19:19:46evanjakedouglas: when we want to GC, all threads are stop, and the object graph is traced, starting at the roots
19:19:50tarcierievan: it's so nice on jruby being able to run one VM and have it use all of the available CPU cores, even for a dinky mostly database bound Rails app :)
19:19:55evanroots include locals variables, etc.
19:21:37jakedouglasright
19:21:58evanthe system doesn't actively track all objects
19:22:13evanit simply knows where to start looking when it wants to find them. :)
19:22:24tarcieriis it weird I've come around to being a fan of threads and blocking I/O?
19:22:25tarcierilol
19:22:34evannot at all to me.
19:22:36evani am.
19:22:48dbussinktarcieri: did you read this? http://al3x.net/2010/07/27/node.html
19:22:53tarcieriyes
19:22:59dbussinkpretty much sums up that there's no silver bullet ever
19:23:04tarcieriindeed
19:23:12dbussinkit's the same story each time, async stuff, nosql, etc etc.
19:23:18jakedouglasis so ready to stop having the i/o discussion because it's always particular to circumstance
19:23:26tarcieriNode is a nifty idea but I really hate callback-driven asynchronous programming now
19:23:36tarcieriwell, I have for awhile
19:24:38evani'm mainly quite tired of silverbulletism.
19:24:48evanand thats not saying anything about a cold Coors.
19:24:53tarcierihaha
19:24:54dbussinkevan: so true yeah :)
19:25:16dbussinkevan: i'd be tired of Coors too if i were you :P
19:25:18evani hear about people saying "you're stupid you don't use node.js for everything"
19:25:26tarcierilol
19:25:30evandbussink: i can't be tired of something I never drink.
19:25:37tarcierithere was this thing we could've used Node for
19:25:48tarcierisort of an async push messaging server
19:25:55tarcierithen defunkt pointed us at the nginx http push module
19:27:25evanthis is an interesting exercise
19:27:30evani'm basically replacing
19:27:42evanGlobalLock::LockGuard guard(...);
19:27:44evanwith
19:27:55evanGCIndependent guard(...);
19:28:24evanso that the GC can run while other threads are inside blocking syscalls.
19:28:34tarcieriaah
19:28:43dbussinkevan: ah ok, cool :)
19:28:56dbussinkevan: also integrates with ffi?
19:29:02evanie, if a thread is waiting inside read(2)
19:29:07evanit's fine for other threads to run the GC
19:29:22tarcieriare you going to support rb_thread_blocking_region? :)
19:29:28evani already do
19:29:30evanand funny enough
19:29:31tarcierio
19:29:36evannow it's just this:
19:29:54evanreturn (*func)(data);
19:29:56evan:D
19:29:58tarcierihaha
19:30:00tarcierinice
19:30:02jakedouglashaha
19:30:23evanbefore it had a GlobalLock::UnlockGuard guard(...) in there.
19:30:35evanI could throw a GCIndependent guard in there
19:30:41evanbut don't need to.
19:31:00evannot GCing while an extension is busy is fine.
19:32:31tarcieriwhat if it's blocking for an indefinite period of time on a system call?
19:33:01evantrue
19:33:06evanshould go ahead and put the guard in there.
19:33:58evansame goes for FFI
19:34:18evanyeah, i'm basically just swapping every place I was unlocking the global lock with guards to go independent
19:34:43brixenso, getting the bts from all the threads in Agent is now going to be super useful for tracking down stuff
19:35:52evanyep
19:35:59evanand we've got that ability easily
19:36:09evanstate->shared.stop_the_world();
19:36:12evan<read backtraces>
19:36:17evanstate->shared.restart_world();
19:36:32tarcierievan: what do you use for the memory allocator for the slabs?
19:36:36tarcierijust libc-provided malloc?
19:37:19evanmy own stuff.
19:37:24tarcierio
19:37:29tarcierithe Erlang people wrote their own too
19:37:33evanthe young gen is allocated with malloc or mmap
19:37:42evanand the slabs are portions of that young gen
19:37:48tarcieriok
19:38:50evanI should grab some lunch.
19:38:57evanbbiab
19:39:10dwaiteevam. upi
19:39:12dwaitere sti[od upi dpm
19:39:17evan?
19:39:31dwaiteevan, you're stupid you don't use node.js for everything
19:39:35tarcierilol
19:39:40evani know.
19:39:41dwaitelike, editing text, brushing your teeth, etc.
19:39:42evani'm an idiot.
19:39:50evanit makes great cat food
19:39:52evani'll say that.
19:40:09dbussinki should make a node.js based online text editor to conquer the software development world
19:40:11evanok, lunch time.
19:40:19tarcieriI really need to blog about the whole async/fiber hype that's going on now :/
19:40:24dwaitedbussink, as long as it supports pointing at textmate plugins
19:40:57dwaitetarcieri: will it just be "you are reimplementing green threads behind the GIL"
19:41:00goyox86dbussink: Nodemate :]
19:41:00dwaite?
19:41:06dwaiteTextNode
19:41:18goyox86dwaite: lol
19:41:20tarcieridwaite: oh, I have lots of complaints... but the lack of real threads is one of them
19:41:34dwaiteI've often been working on say some YAML and thought 'there really should be CSS animation here'
19:49:53tarcieriespecially in light of evan's work on concurrent multithreading here, heh
19:50:21tarcieriwonders if IronRuby supports concurrent multithreading
19:50:31dwaiteprobably? unless the DLR doesn't
19:50:53tarcieriwell it's not something they'd just get for free because the DLR provides it
19:51:02tarcierithey'd need to take steps to ensure the semantics are still correct
19:51:44dwaite"IronRuby maps Ruby's green threads directly to CLR threads, and there is no GIL."
19:51:49dwaiteso says the internet
19:51:51tarciericool
19:52:00tarcieridwaite: thanks, too lazy to Google :)
19:54:26goyox86Node.js uses this technology: http://www.google.com/technology/pigeonrank.html :)
19:56:30goyox86is just kiddin, he thinks Node.js a good project ;) (For ta Node.js fans)
19:56:37tarcierioh I do too
19:56:42tarcieriit's nifty and all
19:56:46tarcieriif you have an appropriate use case
19:56:56tarcieriwe did at one point but we found something that was already written for us :)
19:57:33goyox86tarcieri: yep, nope seeing this as the panacea! , i agree you totally apropiate use case :]
20:00:27dwaiteI think it is a silver bullet, and it will solve all of my problems
20:00:28dwaiteever
20:00:41dwaiteincluding relationship problems
20:00:51dwaiteand certain long-standing unproven mathematical theorems
21:16:51bakkdoorevan: you there?
21:16:58evanyep!
21:53:07sbryantAnything interesting happen lately with rubinius?
21:53:36DefilerBy definition. :)
21:53:56evansbryant: working on GIL removal.
21:54:55goyox86sbryant: should ask if there is some hydra around...
21:55:03goyox86;)
21:56:38sbryantkekeke
21:56:45sbryantevan: neat-o!
21:56:47bougymanevan: hooray!
21:56:52bougymanis it 1.9 yet?
21:56:53sbryantI thought there were issues with it.
21:57:12sbryantWith removing the GIL that is.
21:57:42evannothing inherent in the architecture
21:57:47evanI just always figured it would be a lot of work
21:57:49DefilerIssues like 'evan has to do it'
21:57:50evanbut i didn't really know
21:57:57evanso I decided to dive in.
21:58:03evanDefiler: :D
22:18:09sbryantevan: well, awesome.
22:32:39jakedouglasis there a benchmark laying around that serves to benchmark the gc? (creates lots of objects?)
22:33:17evanmmm, not that I can think of
22:33:17evanno.
22:33:29boyscoutAdd thread::Lockable, add locks to SharedState - 4e30535 - Evan Phoenix (hydra)
22:33:30boyscoutRemove all traces of GlobalLock - dc364a9 - Evan Phoenix (hydra)
22:33:30boyscoutGet stop-the-world for GC working - ba14bec - Evan Phoenix (hydra)
22:33:35evanfor those that are curious ^^^^^
22:33:36jakedouglasHYDDDRRRAAAAA
22:39:26brixenjakedouglas: a couple of the RBS benches create a bunch of arrays and string iirc
22:39:32brixenor really big strings or something
22:40:10brixenbenchmark/rbs/garbage_collection/bm_gc_*
22:40:17jakedouglasthanks
22:40:28brixenthey could be vastly improved
22:40:34brixenas with everything in rbs :)
22:40:53jakedouglasheh
22:41:25brixenjakedouglas: you should use heap dump to grab an object profile 1/2 through a rails request or something
22:41:40brixenuse that to create a similar histogram of objects synthetically
22:42:44jakedouglasyea
22:52:25brixenhttp://gist.github.com/496620
22:52:28brixenyay
22:52:43brixenfair to say perf is scaling with the # of directives implemented
22:53:53brixenthis chunky_png bench http://gist.github.com/493523 is still slow with all directives that it uses implemented though
22:53:58brixenso I need to profile that
22:54:03evanbrixen: btw
22:54:09evannotice you have data in those other columns now
22:54:17evanbecause i'm using getrusage for Process.times finally
22:54:18brixenyep :)
22:54:25brixenI sow your commit, cool beans
22:54:30brixener saw
22:54:34evan:)
22:54:44evangetrusage is more accurate than times(2) which is what MRI uses
22:55:01evanthats why we've got less round numbers
22:55:02evan:)
22:55:07brixencool
23:03:10brixenthanks kernel panic, I wasn't in the middle of anything
23:03:16evanagain?
23:03:17evangeez.
23:03:22brixenyou could have installed the safari update while you were at it
23:03:42brixenfirefox-bin was the current thread
23:03:52brixenbut I swear, it only happens when I run specs
23:04:06evanmaybe a Process#kill spec?
23:04:07brixenmust be some heavy thread switching or something
23:04:23brixencan't see why that'd cause a kernel panic, but who knows
23:04:36evan*shrug*
23:04:40brixenI should just give up on using ff
23:04:52evani've been using chrome
23:05:19brixenI should give it a try
23:07:18evanbrixen: muhahah
23:07:25evani think we've won hearts and minds with RubySpec
23:07:31brixenawesome
23:07:37brixenwhat makes you think that?
23:07:42evansomeone on ruby-core said that if they're going to introduce a Semaphore class, they should be writing specs in RubySpec for it.
23:07:50brixenyay!
23:07:57evani don't know if they will
23:08:02evanbut random people are chiming in.
23:08:07brixenexcellent
23:08:40goyox86brixen: definitely use chrome
23:08:41brixenI'm calling the era before RubySpec the BS era
23:08:45brixenBefore Specs
23:08:50evanheheh
23:08:50brixenmind boggling really
23:09:51brixenevan: this is the extent of unpack in the 1.8.7 tests http://gist.github.com/493054
23:10:07bakkdoorlol, bs era :D
23:10:14brixenit would be hilarious were it not so sad :(
23:10:33bakkdooryeah ..
23:12:05jakedouglasmaybe someone in here can tell me - is there a way to make the MRI source code display sanely in textmate?
23:12:18goyox86evan, brixen: Using BDD/TDD, definitely was a win, it makes a LOT of easy, the diving into the project, when you say "just try to make some spec pass", and we'll check your patch, is like a rock solid foundation :)
23:12:22evanbrixen: http://skitch.com/evanphx/dqwdt/cam
23:12:52evanjakedouglas: try setting the tab width to 4
23:13:02evanshould be a setting in the tray at the bottom
23:14:31jakedouglasnot quite...
23:14:38jakedouglaslooks a little better though heh
23:15:00brixenevan: haha excellent
23:15:18brixenjakedouglas: I think tabs 8
23:15:22brixenthe spaces are 4
23:15:32brixenit's the craziest shit
23:15:33jakedouglasyea tabs 8 worked. thanks
23:15:33brixenever
23:15:52jakedouglasits fiiiine
23:15:55jakedouglas:p
23:15:58brixenhehe
23:18:19evanoh man
23:18:31evanyou know this change is going to be good when you get to use __sync_bool_compare_and_swap
23:19:46brixenyay
23:20:14brixenI think our buzzword compliance is going to scale linearly with cores
23:20:19evani have to think through a bunch code in parallel
23:20:29evanwhich is always a good mental exercise.
23:50:39goyox86i'm getting this http://gist.github.com/496772 when building hydra branch :], i'm pretty sure it's me but can someone help me? :)
23:51:05evangoyox86: it doesn't run the specs
23:51:10evanand i haven't tried to compile extensions with it.
23:51:16evanrake -q build
23:51:18evanonly for now.
23:51:29evaner
23:51:31evanrake -q vm:build
23:51:32goyox86evan: :)
23:51:38evannot sure why you're getting that error though
23:51:44evanthat should work
23:52:03evanyeah, thats even for MRI
23:52:08evanthats not related to hydra actually.
23:52:27goyox86evan: 10-04
23:55:35goyox86evan: same error, let me get Mephistopheles out of my box and try again :)
23:55:49evan:)