Index

Show enters and exits. Hide enters and exits.

08:30:10thehcdreamerHi, I'm trying to install rubinius on mac but I get this backtrace: http://pastie.org/793207 not sure if I'm missing something.
08:45:38somebodythehcdreamer, use pre-build llvm
08:45:53somebody./configure --skip-system
08:45:56thehcdreamersomebody_: ok thanks
10:29:33duncanmvevan: hi
11:04:11duncanmvcd rue/j rails
11:04:15duncanmvargh
11:11:35dbussinkduncanmv: going well, he? ;)
11:39:58duncanmvjust hit http://github.com/evanphx/rubinius/issues/labels/build#issue/86
12:11:02kronos_vanoduncanmv, try to use prebuild llvm
12:25:30duncanmvkronos_vano: I am packaging rubinius so not possible
12:57:12kronos_vanoduncanmv, also you can build rubinius without llvm
12:58:29duncanmvkronos_vano: that is how I had before
14:11:43duncanmvevan: I got the first rubinius package: http://download.opensuse.org/repositories/devel:/languages:/ruby:/rubinius/ , building without llvm for now (issue above) and binaries are not symlinked to /usr/bin yet (I have to find out how to make it use alternatives or any method so they dont clas with normal ruby)
16:19:44tarcieriBinding is silly
16:20:02tarcieriapparently rbx has a non-silly Binding object?
16:20:08tarcieriamasing!
16:20:11tarcieriwith an s
17:13:13evantarcieri: :)
17:13:23slavaevan!
17:13:29evanMr. S!
17:17:56lypanovevan. slava.
17:20:34slavaevan: my epic visual studio port is complete, all gnu assembly is gone
17:20:50slavaevan: I also removed the primitive table. primitives work similarly to rbx now :)
17:21:01slavabefore I had to maintain two lists, and the order had to match up, so stupid
17:34:37rueMorning
17:36:13evanslava: ha! yeah, i learned that sucks a long time ago.
17:36:23evanname => function pointer is a lot easier.
17:48:43BrianRice-work"similar to rbx" means name => fp?
18:06:44evanBrianRice-work: i'm assuming thats what he means.
18:07:22BrianRice-workk
18:08:42slavaevan: yeah, primitives are done with what is a tiny baby FFI
18:09:53evanoh, how so?
18:10:03evanmine are as well, but it's more like a static FFI
18:10:15evandone by infering wrapper code from the signature of the C++ method.
18:11:23slavahi evan
18:11:28evan:)
18:11:33slavaI renumbered my irssi tabs, and keep getting confused
18:11:48evanhehe
18:11:55evanyeah, that happens to me sometimes too
18:11:57slavaits always th esame people in every channel anyway
18:12:04evanyeah
18:12:06evananywho
18:12:38evanyou're right on using one big chunk of address
18:12:46evanit has a bad limiting factor.
18:13:57evani guess in that case
18:14:09evanyou'd have to clamp the size
18:14:25evanand have a hard error when you try to create too many VMs in the same address space
18:15:07evanslava: it appears that by default on my machine
18:15:13slavadoes rbx support mvm yet?
18:15:18evanthe JVM takes up 428M of virtual mem.
18:15:34evanslava: ironically, we were the first to support it
18:15:38evanbut i've since broken it
18:15:43evanfixing it again is easy though.
18:16:19slavaneat. supporting mvm was one of my motivations for switching to c++ but I'm not there yet :)
18:16:48evannot using any global data since the very beginning makes it quite easy. :)
18:17:19slavaevan: our vm objects don't use global data at all, to the point that if you start a new vm it has no way of communicating with the other one. that's he problem :)
18:17:41slavait needs something more efficient than pipes, to take advantage of the sahred address space
18:17:42evanwell
18:17:47slavato pass messages back and forth
18:17:52evanyeah, i coded that up too
18:17:55slavahow did you handle that?
18:17:57evanthere is a manager object
18:18:10evanto handle the multiple VMs
18:18:29evanso, what i did there was the manager object had mailboxs
18:18:31evanone for each VM
18:18:36evaneach VM has a VMid
18:19:03evanand a VM could wait for a "VM message" by doing Rubinius::VM.read_message
18:19:12BrianRice-workhm
18:19:14slavadid you combine VM messages with non-blocking IO?
18:19:17evana pipe was used
18:19:22evanto provide IO blocking
18:19:24evanand waking
18:19:28slavasee what I'd want is to have a green thread in each VM in charge of talking to other VMs
18:19:38evansure
18:19:42slavaand incoming messages from other VMs would wake up green threads, in the same way that pending IO does
18:19:43evanyou use the pipe trick.
18:19:47slavawith one big multiplexer
18:19:47slavaok
18:19:51evanto announce "there is data ready"
18:19:53slavaso you have a pipe and you add it to the multiplexer?
18:19:59evanand you store the data into a buffer that the other VM has access to
18:20:01slavaand store the data elsewhere, in malloced memory?
18:20:02evanin my case
18:20:02slavaok
18:20:07slavathat sounds reasonable
18:20:07evanI used the internal marshaller
18:20:12evanto take a ruby object, turn it into bytes
18:20:12slavais there a pipe between every pair of VMs?
18:20:16evanput it into a mailbox
18:20:26evanwrite to the other VMs pipe, so it wakes up
18:20:34evanyeah, a read/write pair per VM
18:20:47evana VM waits on it's read pipe when it wants a message
18:20:49slavaok, cool
18:20:52evanand a writer does
18:21:00evanwrite(vm[id].write, "!", 1);
18:21:05evanafter it puts the message in the mailbox
18:21:12slavasee, I want to have multi-VM threading, and also have a thread pool in each VM for async ffi calls
18:21:25slavathis is better than native threads that share a GIL
18:21:42evanah, yaeh.
18:22:05slavafarming out ffi calls to a native thread pool, and running green threads, has better properties
18:23:11slavado you think you'll add some form of green threads at some point?
18:23:28evanwell, i'll probably add fibers post 1.0
18:23:30slavawhat machinery do you use for ruby 1.9 enumerators?
18:23:36slavafibers they're called?
18:23:46evanfibers are basically coroutines
18:23:57evana simple version of continuations
18:24:08evanyou'd build a green thread scheduler on top of them
18:24:08slavayeah, and green threads are closely related to coroutines
18:24:09evanif you wanted.
18:25:56slavaevan: and then you'd need soome code to turn blocking IO API calls into yielding to the green thread scheduler and doing non-blocking IO for all green threds at once
18:26:22slavaevan: if you make the green threads efficient enough this will be a big performance boost for network servers that handle large numbers of connections
18:26:43slavaevan: and the code won't have to be all contorted like anything that uses twisted python or similar
18:26:49slavaprogramming directly for non-blocking io is teh suck
18:27:09slavajruby will never have this :)
18:29:27jptixslava: i think you'll need to repeat that later :)
18:30:49BrianRice-workjust have him blog about it
18:31:08BrianRice-workI've been looking into this for a bit; might make it my next priority
18:33:26slavaBrianRice-work: its a shame that no books about VMs discuss implementation of green threads
18:34:17BrianRice-workslava: yeah, Iain Craig's book gets a tiny bit into it, but not in a truly helpful way. it's typical academic-style book publishing, though.
18:35:40evani'm back
18:35:42evanwhat did i miss.
18:35:49evanwhile freenode freaked the fuck out.
18:36:03BrianRice-workhm I don't have that book on my desk. I think I sent it back to my home bookshelf
18:36:42BrianRice-workevan: slava had a few more comments on green threading and blocking i/o
18:36:53evangotcha.
18:37:00evanwas it 'DON'T DO IT!"
18:37:01evan:)
18:37:01BrianRice-work"<slava> evan: and then you'd need soome code to turn blocking IO API calls into yielding to the green thread scheduler and doing non-blocking IO for all green threds at once
18:37:02BrianRice-work evan: if you make the green threads efficient enough this will be a big performance boost for network servers that handle large numbers of connections
18:37:02BrianRice-work evan: and the code won't have to be all contorted like anything that uses twisted python or similar
18:37:02BrianRice-work programming directly for non-blocking io is teh suck
18:37:03BrianRice-work jruby will never have this :)"
18:37:33evanthats what MRI does
18:37:43evanso MRI already has a leg up on twisted
18:37:44evanin that regard.
18:37:52slavabut does it scale in practice?
18:38:18slavaanyway, a hybrid m:n threading system would be ideal
18:38:20slavaif you got rid of the gil
18:38:31slavabut supported green threads for non-blocking i/o's sake anyway
18:38:42evanwell yes
18:38:48evani've got a few moves to make
18:38:52evanone is removing the GIL.
18:39:04evani've already been making steps in that direction
18:39:09evanbut probably won't finish before 1.0
18:39:29slavagreen threads sounds like something that will be a natural next step once you implement the fiber api
18:39:40evancertainly we can play with it.
18:39:47slavaonce your vm can play with its own call stack
18:39:56evanI should check out how sun's MxN setup works
18:40:26slavaI thought the jvm was always 1:1?
18:40:33evannot in the jvm.
18:40:34evanin solaris.
18:40:37slavaoh
18:40:39slavahmm
18:40:40evansolaris has (or had) MxN
18:41:27slavaI presume the userspace component is similar to how threading is implemented in libc on some bsds
18:42:12slavaI'm not sure how it works with blocking system calls
18:42:25slavado they block all threads on the 'N' side of the equation?
18:42:33slavahow can it be truly pre-emptive?
18:42:56evanright
18:43:15slavait seems unusual to attempt to do transparent M:N threading at the OS level, since you're papering over the semantic differences etween real threads and what you cook up in user space
18:43:15evani'm not sure if balances the userspace threads among the kernel threads
18:43:28evanor if it implements a userspace scheduler that can "block" a userspace thread for IO
18:43:28slavamaybe in solaris, all syscalls have a non-blocking version?
18:43:41evanright, becuase if you do
18:43:43slavaand so the threading lib implements a big multiplexer and scheduler
18:43:47evansyscall(0x7f, ...);
18:43:51evanthen you block.
18:43:54slavayeah
18:44:00slavahmm
18:44:06evanand unless libc has a special syscall function that doesn't block
18:44:09evaneven so
18:44:18evanthere is code that uses RAW syscalls.
18:44:22slavathat would add so much overhead if all calls were non blocking
18:44:23evanwhich libc can't see.
18:44:29slavayeah
18:44:45evanso I dunno
18:44:49slavamaybe they just had a separate API forgreen threads? do you have a reference for this?
18:44:52evanmaybe there is some kernel support to assist
18:45:02evansort of like how the page tables work
18:45:11evanyou manipulate them at the higher level
18:45:19evanso the lower level can read them and react.
18:45:37evanlets see if i can find it.
18:46:22evanha
18:46:23evanhttp://www.sun.com/software/whitepapers/solaris9/multithread.pdf
18:46:25evanpage 18
18:47:49evanso they probably had too many problems.
18:47:58evanand if they were swappable
18:48:04evanthen it was surely only at the libc level.
18:48:14evanso there was still the ability to block a kernel thread
19:37:48boyscoutCI: rubinius: 6b9b7ad successful: 3024 files, 11752 examples, 35958 expectations, 0 failures, 0 errors
19:38:21evanthat was off.
19:38:22evanodd.
19:50:38devinushas anybody managed to use rails 3 on rubinius?
19:51:15evani've setup a basic app
19:51:18evanusing sqlite3
19:51:27evanhaven't gone much beyond that.
20:20:35kronos_vanoevan, I've sent pull request. Do you see patches?
20:20:58evansorry i haven't gotten to them
20:21:28kronos_vanook
20:22:25evanthis commit
20:22:25evanhttp://github.com/kronos/rubinius/commit/ee9b5d7c835fc1f9b99cb446023498f0dca13e49
20:22:29evanis incorrect
20:22:39kronos_vanoexplain please
20:22:41evanwe don't allow changes to specs in commits with other things.
20:22:46evanyou must do them as seperate commits.
20:22:54kronos_vanook
20:22:54evanthats allows us to sync spec/ruby
20:23:14kronos_vanoI already sent patch to rubyspec
20:23:17evank
20:23:24evanwell, you said these are patches as well?
20:23:29evanin issues
20:24:45kronos_vano#162 #163
20:25:38evanplease tag them as patch
20:25:47evanthat makes them easier to sort out
20:25:52evani'll apply them later today
20:26:42kronos_vanook, tnx
20:45:43slavaevan: lets make an XML virtual machine with a self-hosting XML parser
20:45:52evansounds like fun!
20:46:01slavagenerate machine code from assembler templates with XSLT
20:46:14slavathe name sounds bad-ass
20:46:20slavaXML Virtual Machine enterprise edition(tm)
20:47:10evanXVM
20:47:37evanhow many levels deep does the XSLT transforms need to be?
20:47:42evani'm thinking at least 5
20:48:00evanmachine code optimizations via XSLT transform
20:48:10BrianRice-workow this just makes my brain hurt
20:48:29evanwhere's your sense of adventure!
20:49:31BrianRice-workmaybe if it could be macro-generated from scheme or something :P
20:50:29slavaparrot should switch to xml bytecode
20:50:38slavajust when you thought it couldn't get any more absurd
20:51:25slavathe trick is to make an xslt to machine code compiler, in xslt
20:51:25BrianRice-workisn't aware of parrot absurdities, but he's chosen not to pay attention to it...
20:51:28slavaand use it to compile itself
20:51:37tarcieriObject#id IS deprecated; use Object#object_id OR ELSE.
20:51:38tarcierilol
20:51:50evan:)
20:52:04tarcierior else you are likely to be eaten by a grue?
20:52:15evancourse.
20:52:20tarcieriawesome
20:53:50tarcieri>> Marshal.dump Binding.new
20:53:50tarcieri=> "\004\bo:\fBinding\000"
20:53:52tarcieri:O
20:53:55evanheh
20:54:00evansupported!
20:54:04tarcieriyay!
21:02:17slavatarcieri: can your company give me a few million to design enterprise xml acceleration cards?
21:02:29tarcierirofl
21:02:50slavathe special coprocessor uses proprietary anglebracketscan(tm) technology to parse xml faster than a cpu
21:03:04tarcierisweet!
21:03:06slavaall using the xml virtual machine of course
21:03:11tarcierithat's sure to make JBoss boot faster
21:03:27slavayeah, just convert the JBoss JAR file into XML
21:03:54slava<jar name="JBoss.jar"> ... base 64 encoded data follows
21:04:53tarcieriJBoss needs to parse a few billion lines of XML in order to boot already
21:04:54tarcieriheh
21:05:24rueHuh, I wonder if it was brixen that went through the Picasso at MoMA
21:05:26slavadoes it do DTD validation every time too?
21:06:05slavaan XML virtual machine will compile DTDs to native code using tracing JITs
21:06:22slavayou could really get some suits excited about this
21:06:54tarcierilulz ohjeezus
21:07:25slavaI think we should tunnel all TCP/IP traffic through SOAP
21:07:59slavathis way firewalls will only ever have to open port 80, it will be teh secure
21:08:20DefilerJSON over UUCP over SOAP
21:08:28tarcierilulz uucp
21:08:34BrianRice-workpunchcard over JSON
21:08:54tarcieriafter using JBoss I am really impressed by the whole application platform concept and wish there were some sort of Ruby equivalent
21:09:16tarcieriJBoss itself is still and ugly monsterous tentaclebeast
21:09:30slavais hot code reloading pretty much non-existent?
21:09:48tarcieriit's kind of there
21:10:02DefilerMy favorite JBoss game is watching it boot and seeing how many acronyms I recognize
21:10:05tarcieriyou copy a "WAR" file into its deploy directory and it magically notices it somehow and deploys it
21:10:07BrianRice-work"people hate J2EE. ruby guy invents Rails as alternative, markets it like crazy making Ruby popular. Ruby user 5 years later decides that JBOSS is cool, ports it to Ruby. Ruby becomes anti-marketed, loses prominence. Javascript inherits the earth."
21:10:09slavaI've read about it taking 30 seconds to bounce an app
21:10:11rueDefiler: Why would you use JSON if you are already using XML?
21:10:15tarcieriDefiler: hahahahahahahaha no doubt
21:10:24tarcierislava: yeah that's pretty much the case
21:10:42slavaJBoss prints more messages than the linux kernel when it starts up
21:11:02Defilerrue: Well of course your company mandates that all messages be in XML format, so you do <json> </json>
21:11:05tarcieriyeah watching JBoss try to initialize a data source spews literally a few thousand lines of logs
21:11:26slavaso why do you want this for ruby?
21:11:36DefilerWait, you weren't kidding about liking JBoss?
21:11:36tarcierioh I can't initialize this data source yet! I need the AbstractDataSourceAdapter!
21:11:37DefilerWhoa.
21:11:51tarcieriDefiler: liking the application platform concept, not JBoss
21:12:06Defilergem bundler is all the application platform I feel the need for
21:12:08tarcieriDefiler: I have database failover! :O :O :O
21:12:23tarcieriit was actually easy to configure!
21:12:32evanTHIS.. IS.. REPLICATION!
21:12:46tarcieriwell, yeah, but... client-side support
21:12:47tarcieriheh
21:12:58DefilerYou have database failover that will work as long as none of your databases actually fail
21:13:03DefilerIs generally what that means :)
21:13:11tarcieriwe did a test today
21:13:14tarcieriit worked just dandy
21:13:28slavaDefiler: `j2ee technologies only have to give the impression of actually working
21:13:33DefilerWas your test filling up the disk on one of your databases?
21:13:35DefilerHeh
21:13:47tarcieriit was literally unplugging the database server from the network
21:13:54tarcierior rather, one of them
21:14:02slavawhat if its plugged in but the data is corrupt on one of them?
21:14:11tarcierithen you're fucked I guess?
21:14:11tarcieriheh
21:14:16Defilerexactly
21:14:19Defilerfailover is domain-specific
21:14:21slavaso what's the point of having it?
21:14:27slavaeither way, if a disk dies, you're fucked
21:14:30tarcieriin case the psu dies?
21:14:35tarcierior it doesn't boot for some reason
21:14:35slavaexcept if you have two disks and you don't know which one is right, youre fucked too
21:14:40tarcierior the network card dies
21:14:43tarcierior mysql won't start
21:14:50tarcierior it overheats and crashes
21:14:50slavaok, but if one gets corrupted
21:14:56tarcieriyes it won't solve corruption
21:15:02tarcieribut you could make the same argument for RAID
21:15:02slavaif you know which computer is bad, then its useful
21:15:04slavayou take it offline
21:15:06tarcierithat doesn't mean RAID is useless
21:15:08slavabut if you don't know, it won't tell you
21:15:26slavawell, as Defiler noted, there are better ways of doing this
21:15:39tarcieri*shrug*
21:15:42tarcieriit solves a number of problems
21:15:46tarcierinot all of them, but a number of them
21:16:04slavais this actually ahibernate feature?
21:16:05tarcieriany that result in a total node failure
21:16:11tarcieridunno
21:16:19tarcieriit's a feature of a JNDI datasource, I guess?
21:16:24tarcieriJava acronyms confuse me
21:16:33slavajndi, wtf is that
21:16:44tarcierijava name doohickey, inc
21:17:01DefilerJava Naming and Directory Interface, I think?
21:17:05slavasee, ruby doesn't need most of this crap
21:17:07tarcierisure!
21:17:21tarcieriyeah Ruby could do it with 5 billion fewer layers of abstraction
21:17:44slavajava apis are always so generic
21:17:58kronos_vanoCan anyone explain how to tag tickes on github?
21:18:00slavaeverything is a wrapper around somethingelse
21:18:09Defilerkronos_vano: I couldn't figure it out
21:19:39evanyou click the check boxs next to the ticket
21:19:50evanthen you click the "down" arrow on the tag and select "apply to selection'
21:20:19evangreat
21:20:21evanit appears to be broken.
21:20:57evanoh, there it goes.
21:21:05evanyeah, it seems to not work sometimes though
21:21:27DefilerI had trouble getting the checkbox behavior, vs. it taking me to the tickets that had those tags
21:21:42evan*shrug*
21:21:45evani need lunch.
21:21:46evanbad.
21:22:05slavaevan: when you generate code with llvm, does each function you generate have the same signature, at the ABI level?
21:22:54evanyes
21:22:55slavawhat do the function pointers look like when you call compiled code?
21:23:06evanthey all have the same signature.
21:23:10evanok, bbiab.
21:55:46khaasehi
21:56:01khaaseanybody seen something like this: http://gist.github.com/286298
21:56:23Defilermake sure you don't have any old .rbc files around
21:56:33DefilerThat's what comes to mind, at least
21:57:20khaaseyeah, you're right. really sucks rvm doesn't cope with that. guess that was the issue the last time, too.
21:57:57Defilerrvm may need to be hitting up a 'clean' task somewhere I guess
22:09:53devinusevan: how useful would a grid of packages that do/don't work in rubinius yet be?
22:10:12evansure, seems useful.
22:12:07jvoorhisevan: hi
22:12:22evanhello
22:13:04jvoorhiswithout looking at what i have so far, can you tell me how you'd expect ruby-llvm to wrap LLVM types?
22:14:21jvoorhisi didn't have any design going in, and i just realized i'm going to have to tidy it up
22:14:55jvoorhisone thing i want to support easily is managing parallel FFI::Struct and LLVM::Struct declarations, if feasible for a first release
22:15:49evanew.
22:15:50evanno
22:15:52evandon't bother.
22:15:54evanthats not useful.
22:16:01jvoorhishm
22:16:08evanLLVM types should be normal objects
22:16:31evanunrelated to any of the FFI type stuff
22:16:39jvoorhismy use case would be for sharing PortAudio's user_data pointer between a ruby app and a jit-compiled callback
22:17:13ruedevinus: Very useful; reserve isitrbx.com :)
22:17:15evanyou could make something that takes an FFI::Struct
22:17:24evanand creates an LLVM::StructType for it
22:17:34evanbut i wouldn't go much beyond that.
22:17:41evan"unifying" them isn't useful.
22:17:52evanand will just cause more grief than be helpful
22:18:11devinusrue: good idea, thanks
22:18:32jvoorhisevan: that's a good idea
22:18:39jvoorhisi'll play with it
22:18:55evangiven the trivial nature of FFI::Struct
22:18:58evanthat should be easy.
22:19:10devinusi'm sort of just a dabbler in the ruby scene, but rubinius excites me, and i keep coming back to check on the status. it looks freaking fantastic. i'm amazed that most packages are working out of the box for me
22:19:26jvoorhisevan: yeah, ffi's reflection is probably enough
22:19:52devinushas there been any talk of one day making rubinius the de-facto ruby?
22:20:14devinus(or i guess the community "blessed" ruby?)
22:20:29rueOnly in the cabals
22:20:47evandevinus: we're amazed at how much runs on it too!
22:21:06evanhell, i'm amazed everytime eval is used and it actually does the right thing.
22:21:11evanso i'm easily amazed perhaps.
22:21:43evanhaving written Kernel#eval a couple of times though, you can understand my amazement :)
22:22:12evandevinus: talking about rbx being de-facto or such is, in my mind, outside the project itself
22:22:17evanif we make it kick enough ass
22:22:26evanthen things will answer themselves.
22:25:57devinusevan: indeed, i love it. you've done some great work. the contributors too
22:26:30slavarubinius will kick major ass once evan completes The Plan(tm)
22:26:44evanhaha
22:26:51evanis that like the cylon plan?
22:27:50slavaits more like, you'll just keep plugging away at it, and eventually it will have every optimization
22:28:09evanyep.
22:28:57slavamy new blog post has a summary of the internal ABI I use, if you're interested
22:30:03evanoh yes.
22:30:05evanurl?
22:30:13slavahttp://factor-language.blogspot.com/2010/01/replacing-gnu-assembler-with-factor.html
22:30:16evanthe one on assembler?
22:30:17evangotcha
22:34:41devinusdoes rbx irb not read .irbrc?
22:34:53devinusi remember awhile back it did
22:37:09evanit reads it
22:37:17devinusokay
22:40:59evanslava: nice
22:41:24evanslava: something you didn't mention that I always think about is GC pointers in C++ frames
22:49:33kronos_vanoevan, I couldn't tag tickets, because only owner and collabs cat do this :( My patches is #162, #163 and #165.
22:49:51evank
22:57:17slavaevan: well, it knows which frames are C++ frames and which ones are factor frmes
22:57:31slavato find roots in C++ frames, it looks at an stl vector of addresses
22:57:36evanyeah
22:57:37evani know
22:57:41evanbut the reader might not
22:57:42evanwas my point.
22:57:47slavaah
22:58:47slavathanks for going over it
22:58:52evannp.
22:58:56slavadid the calling convention make sense otherwise?
22:58:58evansweet, i fixed a deopt bug.
22:59:12evanslava: i think so
22:59:27slavaits got those funky pseudo-parameters for quotations and tail calls
22:59:33evanone thing that might make it clearer so to, at the boundary of factor and C++
22:59:37evanmake a diagram
22:59:42evanthat shows what should be in each register
22:59:45evanand what should be on the stack
23:00:01evanthose diagrams always help me when i'm looking at call convention stuff
23:00:06slavayeah
23:03:10slavaI'm working on deopt right now too and plan on blogging about the impl
23:06:48evanoh? whats your plan?
23:07:11slavaI just need to fix some bugs
23:07:12evanare you doing it because you want to inline words other than ones marked "; inline" ?
23:07:33slavano, either way its the same problem
23:07:58slavastuff gets redefined, has to recompile minimum amount of stuff in order to make everything work
23:18:36evanah, gotcha.
23:31:46evanslava: what is in your code heap?
23:32:05slavablocks of native code
23:32:30evanwell, sure, i mean, whats the form of them
23:32:35slavaoh
23:32:36evani assume they're structured in some way
23:32:47slavaa header with a code_block struct
23:32:52evansome metadata around the machine code
23:32:54slavafollowed by machine code
23:33:13evanand it obviously doesn't compact
23:33:16slavametadata is a relocation table, parameter table, and a pointer back to the owner object whose compiled definition is this code block
23:33:19slavait does
23:33:28evanoh right, with reallocations
23:33:50evanok, so the pointer back to the owner object
23:33:51slavacode can get moved around, and it hasto update return addresses in the code heap as well as machine code itself
23:34:06evandoes that mean that you run write barriers there?
23:34:07slavas/code heap/call stack/
23:34:28slavaevan: yes, either one of the owner object, relocation table or parameter table could point into young space
23:34:46slavaand the code block's machine code itself could reference literals in young space
23:34:54evanso you update the card table for those?
23:34:56slavaso there's a write barrier in the form of a remembered set
23:34:59slavanope, just an stl::set
23:35:00evanah
23:35:02evanok
23:35:06slavasince its empty 99% of the time
23:35:12evanright
23:35:15slavawhen new code is compiled, the new blocks end up there
23:35:20slavaand rapidly get cleared out after a few cycles
23:35:26evanright
23:35:29evanas things are promoted
23:35:34evanwhen do you run the GC on the code heap?
23:35:49slavaI always collect the oldest generation of the data heap together with the code heap
23:35:54slavaif one fills up, both are collected together
23:36:10slavathis is because they reference each other circuarly and doing both at once is the only way to collect unreferenced cycles
23:36:18slavaeach word references its code block which references the word
23:36:18evangotcha
23:36:29evanright
23:36:35evanalso, not doing them both complicates the remember set
23:36:36slavaso the mark stack during a full gc is a mix of data heap objects and code heap blocks
23:36:44evanand the idea is that you do both less often
23:36:58evanhow often do you do a old gen collection+
23:36:59evan?
23:37:11slavawhenever old gen or code heap fills up
23:37:23evansure, i mean normally
23:37:31slavathat's the only case
23:37:37evanevery few seconds? if there is a lot of churn?
23:37:42slavaoh
23:37:51slavano, much more rarely than that hopefully
23:37:53evani just mean the observed time
23:39:01slavafull collections are very rare because almost no object survive into tenured space
23:39:24slavathe worst pathology I've seen is with the PEG library
23:39:37slavawe have a javascript parsing bnechmark that spends 50% in GC
23:39:52slavaand half of the GC time is 4 major collections
23:40:00evanaah
23:40:03slavabut most things won't trigger it at all
23:40:07evanright
23:40:20evansomeone wrote what they thought was an Array#<< benchmark
23:40:26evanbut it was actually a GC benchmark
23:40:31slavawhy is that?
23:40:41evanit did, basically
23:40:42evanary = []
23:40:56evan10_000_000.times { ary << Math::PI * 2 * 4 }
23:41:13slavaso this boxes some floats and stores into the array on each iteration?
23:41:17evanactually, i think it was more than 10 million times
23:41:19slavaand then every minor GC scans the whole array?
23:41:30evanyep.
23:41:49slavayeah, card marking fixes that and if you use float arrays there's no boxing at all
23:42:12evanof course it's different if I change the code :)
23:42:19evanbut card marking wise, yes.
23:42:38evani'm thinking about adding a seperate remember set for Tuples
23:42:49slava: foo ( -- ) 10,000,000 [ pi 2 * 4 * ] float-array{ } replicate-as drop ;
23:42:54evanwhere the address of the slot in the tuple is recorded
23:42:55slava0.07 seconds
23:42:58evanrather than the address of the Tuple
23:43:10evanusing a float-array is cheating.
23:43:27slava: foo ( -- ) 10,000,000 [ pi 2 * 4 * ] replicate drop ;
23:43:31slava0.09 seconds
23:43:54evannice
23:43:59evanyou're using float boxing, right?
23:44:05slavain #2, yes
23:44:12slava#1 allocates no memory other than the float array at all
23:44:31slavaactually, that's incorrect, in both cases pi 2 * 4 * is constant-folded down to a literal float
23:44:34slavathere's no boxing in either case
23:44:35evandoes replicate create a new array or just loop?
23:44:52slavait creates a new array calling the quotation to geerate each element
23:45:00evanhow is there no boxing if you're creating an array with 10 million generic elements
23:45:05slavain this case it returns the same thing on each iteration, so it works like << in a loop
23:45:05evanyou should have at least 10 million boxes
23:45:26slavaeach element points to the same boxed float, though, because of constant folding
23:45:32slavait computes 2*4*pi at compile time
23:45:38slavaand saves the result as a single boxed float
23:45:42evanok
23:45:45evanthe real benchmark did
23:45:46evanary = []
23:45:49evan10000.times do
23:45:53evan|i|
23:45:58evan4000.times do |j|
23:46:03evan ary << Math::PI * i * j
23:46:03evanend
23:46:04evanend
23:46:06evanthat might still fold
23:46:09evanbut they'll all be different
23:47:52slava40 million boxed floats? I'm running out of memory
23:48:28slavafor 1000x1000, it takes 0.7 seconds
23:49:06slava0.04 seconds with unboxed floats
23:49:14slavaso once you're actually calculating, unboxing makes a bigger difference
23:49:55evani'd like to compare apples to apples, so yeah, the unboxed array is cute, but not nearly the same
23:50:01slava10,000x4000 is 0.5 seconds with unboxed floats, runs out of memory with boxed
23:50:29slavaso a boxed float is 16 bytes
23:50:34slavaplus 4 bytes for the reference
23:50:52slavaso that benchmark needs 800mb of heap
23:51:12evanlooks like the benchmark was 10000 * 400
23:51:41slavaah
23:51:44slavaand how fast does rubinius do it in?
23:51:49slava: foo ( -- ) 10,000 iota [ 400 iota [ * pi * ] with map ] map drop ;
23:51:56slava0.5 seconds, with boxing etc
23:52:31slava90% of that is GC
23:53:04slavayeah, much cuter with unboxing...
23:53:44evanlets see..
23:54:34evanlooks like 5.2s for rbx
23:54:42slavaalso 90% GC?
23:54:52evanactually, no, it's worse than that
23:54:54evanlets see.
23:54:59maharghm. Is it still accurate that rubinius isn't thread safe due to borrowed MRI code as stated in the README?
23:55:13evanmaharg: wow
23:55:16evanthe readme says that?
23:55:17evanwhere?
23:55:51maharg5. Goals, first point
23:56:16evani.. dunno who wrote that
23:56:20evanit's entirely wrong.
23:56:33mahargheh. I thought it seemed unlikely.
23:56:53evanit's been threadsafe for years.
23:56:57mahargbeen a long time since I looked in on rubinius' progress, though. Come a long way.
23:57:32slavaevan: what kind of time are you getting?
23:57:43evanlooks like 8.2s
23:58:02evanjust the one float * adds 3s
23:58:08evani haven't done much to make those fast
23:58:14evanand boxless.
23:58:19evanon operations