Zaki.asia

Startup Weekend Notes

dezso.zoltan@gmail.com (Zoltan Dezso) — Mon, 28 Nov 2011 01:55:18 -0800

I went to Startup Weekend Tokyo in November 2011. Below are my unrefined notes about this experience.

Day 1 - Pitches

Half Baked

Things started off with a game called "Half Baked". There is a list of - usually weird - words from which each team choose two. Those two words will be the name of the teams' imaginary companies for which each team must come up with a pitch. In five minutes (time pressure is a recurring theme at Startup Weekend - just like in real life).

This game is intended to represent Axiom #1:

It is not the Idea but the Team that makes a successful startup.

Now I'm of the opinion that that proponents of this axiom might have the causality relationship upside down: the most important part for any startup is the idea. However a good team is necessary to find that idea (unless you already have a bulletproof idea, which you probably don't)

1 - The IDEA is the most important part of a startup. A great TEAM is the best way to it.

When I threw ideas out in a brainstorming fashion, those were taken as fully formed ideas and were not challenged. I can't just write this off as a cultural curiosity. Japanese do tend to be conflict-averse to a pathological level, but these are surely the best and are pioneers. My... uhm... half-baked idea was selected almost unanimously in a team vote.

Five minutes preparation time went away pretty quickly and at the end we didn't really have the pitch down, which put the guy who volunteered to pitch in an unenviable tough position. And to my utter surprise he built a case out of thin air - he pulled it off spectacularly.

The pitch

Sixty seconds is not a lot of time to introduce the problem and the proposed solution in a memorable way. This task is not helped by the fact that everyone pitches and sooner or later (and let me tell you that it's sooner) pitch fatigue sets in. Out of the 60 or so pitches, after a week, I remember maybe 3 or 4 and one of them is the pitch of the above guy that I was specifically keeping an eye on and his cause is helped by the fact that he got selected and that he failed miserably at coming up with even a hint of project to do and the team dissolved on the first day (although later I understood that in fact he only wanted to practice the pitch and as such did a brilliant job).

2 - More than one pitch causes pitch fatigue. Make sure you have undivided attention.

There were two problems with my pitch. The first is a cosmetic issue, namely that I was literally shaking from anxiety. I suck at public speaking but wasn't aware of just how much. And I wasn't aware how much that is not a unique problem. On the one hand, probably no one else noticed and on the other, I was looking at others and most everyone was nervous on the stage. I am glad I pushed myself to do it.

3 - Everyone sweats it. You just have to push yourself.

The second issue is that I did not articulate the solution as well as I did for the problem. I had 13 seconds left after I introduced the solution and made the split-second decision to not risk botching it by making a clearer connection between problem and solution. Probably I should have practiced more beforehand, but the thing I noticed is that with all these problems, my pitch was pretty decent compared to the other 59. It didn't make the cut, but came closer than I hoped - to the point where I was actually afraid it might get selected and I have to lead a team (got to third place in the online voting).

Looking back, I think I actually sounded confident and brought up a problem that everyone actually had. I had been mulling over this project for a very long time now (this is unfortunately my MO) and had a fairly good idea of what I am trying to achieve. From the people who got selected, sounding confident and telling a story that resonates with the audience are the most important parts of the pitch.

4 - Tell a story. Sound confident. Involve the audience.

As I mentioned, I had things thought through before. I thought about target segments, business models, revenue streams even pivot options if things go awry. Overkill? Definitely. People who got selected with a few exceptions had absolutely no idea what they are going to build and how they are going to succeed. As it turns out, two teams dissolved shortly after getting selected. Talk about high failure rate in startups :)

5 - A good pitch is a poor indicator of future success.

In fact, thinking about it, probably the most successful startups are the ones that don't really pitch, because there is no way they could not not do what they are doing.

Day 1 - Team Building

Now, that the 12 teams have been selected, there was a little time to go around and decide on which team to join. At some point I got the feeling that none of the teams are actually very good. There were 3 or 4 I didn't really consider from the start. Another 3-4 that didn't really have the slightest clue of what they are going to do. In the end I decided to not worry too much about it and just have fun.

6 - Startup Weekend is NOT a realistic simulation of a startup. Just have fun.

While in the end I think I got to know great people, out of the 90 or so people in the room, I would not consider starting a company with maybe 85 (would still work with most, just didn't feel a connection that's strong enough for starting a company together). Startup Weekend foregoes something I regard one of the highest priorities in early startups: getting the best people. It'd be a good idea to tailor the pitch so that the best people get interested in your idea.

7 - Get the best people

Day 2 - Discussions

The first half of the second day is when the team really gets together for the first time and discusses what they are going to do. I was sort of lucky to join a team that mostly knew what to build. Even with that advantage, it takes a long time to get on the same page.

8 - Communicate early and often.

I said I was sort of lucky to join a team that mostly knew what to build. I lied. One of the most important ingredients was missing, which lead to a lot of rethinking later on: the business model. As a corollary, the target segment and the revenue model were also up in the air. Free from the constraints and pressures of Startup Weekend, this would be a great side project that might or might not become a full business, but that's not what Startup Weekend is about.

Startup Weekend is about finding/founding the startup that will change the world. A side business or and interesting idea will not win regardless of how feasible it might be as a real business. After a lot of thinking, I believe that our team would have been able to build a real company on the idea. We executed pretty well on a small problem (being one of the really few teams that actually had some sort of product to show) and that could grow into a profitable venture. But it's not something that would change the world.

9 - There is a fundamental difference between a project and a startup.

It was surprising to learn that no one in my team knew the Lean Canvas or personas or customer validation etc. I was sweating these things when preparing for my pitch so at least could bring up these tools. In the end though, probably we went into production mode too early, which bit us in the final presentation.

Day 2 - Development

We went into production mode. Or at least I did. That's what I enjoy most. Building things, working out issues, coming up with solutions to problems, optimizing, tinkering, combining. This is what I'm best at. One of the side quests I had for Startup Weekend is to work with other brilliant developers (and as a side effect sort of validate my project). I could have worked with more people, but in the end, this side quest was pretty successful.

I had started working on my pitched project before, so in fact had a working application skeleton that I'd just import and customize for this project and we were up and running with complete facebook integration in under an hour.

The thing is, it didn't matter. I am about halfway through Eric Ries' Lean Startup and this is one of those classic cases where we wouldn't have needed to actually create anything. I think that's one of the failings of Startup Weekend Tokyo GEW specifically. I can't blame the organizers though, because this being an international startup battle, it is a lot more important to have a solid idea that can be elaborated on and then presented in Singapore next year than to create a small startup. Hopefully the next one will have a different atmosphere where creating a real product has a higher value.

However, doing all that development gave me a lot of advantages that I wouldn't have had had I not done it and just thrown together some sketch.

I had an insane amount of fun
Did I mention, this is what I enjoy most?
I could work with other great developers
Which in turn was also adding to the fun factor. That's the sort of atmosphere you want to have in your startup.
I realized I'm a good developer
From time to time impostor syndrome gets the better of me and it is great to be reminded that it's not real.

As a side-effect, my teammates tried to nominate me as the person who worked the hardest in the team. If only they knew just how much fun I had in the meantime :)

10 - Having fun is really important. Developing is fun for developers.

11 - When you are having fun, you can pull off impossible-looking feats.

Day 3 - Crunch

The first half of the third day is crunch time leading up to the final presentations. At this point people are groggy and the fun is starting to dissipate, so we just had to put our heads down and do what we had to do.

This is the time where we started having second thoughts about the fundamentals. I thought that that was not a very good position to be in, but it seems that other teams were also going through this phase.

12 - Everyone has doubts just before the end. Just push through.

Day 3 - Final Presentation

Final presentation is the most important part of Startup Weekend. The pitch is just a tool to get selected and the development is just a tool to put together something that will get the attention of the VCs. Everything rides on how the final presentation goes.

Most teams didn't even start creating a product at all (they were busy arguing about the business). The audience-winning team and the final winner team did not start to create a product.

What they had though is a compelling story. The presentations before the winning team were all so-so but when the winner presented, they were something else altogether. It was probably not fair to the other teams (the winning team was basically the extension of an already existing and successful company). The presentation though... people suddenly started thinking "Steve Jobs keynote" and it wasn't very far.

During their QA with the VCs, I had the most visceral validation of what I once read somewhere (paraphrased):

Tell your weaknesses up front. Tell your strengths at the end preferably when prodded for.

Things went down approximately like this:

VC: OK, but what will you do if <other, already successful company in the same field> does the same thing? XY: Oh, that company happens to be my company.

Boom. Won.

There was a long deliberation afterwards where I'm sure the other teams were considered in earnest, but the sheer force of this exchange is hard to ignore. It's hard to understand from reading about it, but it was visceral; it was powerful. Note, that he wasn't bragging about this, wasn't even going to mention in his presentation, but when the VC asked about it, that one line had all the more power. Combine this with a very well executed presentation and at that point I literally forgot what the presentation is actually about... hand me the pen, where do I sign the check?

On the other hand, we had the start of a product, but lost because of two things.

The idea wasn't big
The idea wasn't solid

That the presentation wasn't too good is just a result of these two.

13 - Make sure you have a compelling story. Product is secondary.

Day 3 - General

One thing I think I didn't do enough of is talk to other teams. Part of it is because I had too much fun working on the product. Part of it is that I'm not that good at networking, but then again now I know that most people aren't either and it's just one of those things that you need to do against all fear.

14 - Talk to everyone.

Am I going to go to the next Startup Weekend? Almost certainly, there are a lot of things I still want to try in the pitch, maybe get selected and have different problems next time. Then again, maybe next time I'll make it about connecting with others rather than concentrating on a project.

So do I recommend going to a Startup Weekend? Yes. Is it really like a startup? I don't think so. Still, it's one of those experiences that are worth trying out.

Rails 3 javascript_include_tag

dezso.zoltan@gmail.com (Zoltan Dezso) — Wed, 10 Nov 2010 01:55:18 -0800

The below issue is already fixed in rails.

After installing the jquery_ujs driver for a Rails 3 app, today it started acting up when I tried using the Cycle jQuery plugin. Specifically, the following code in application.js would create twice as many pager anchors as expected.

$(document).ready(function(){
  $('.featured').cycle({
      fx: 'fade',
      timeout: 5000,
      speed: 600,
      pager: '#featured-controls',
      pauseOnPagerHover: true,
      pause: true,
      pagerAnchorBuilder: function(idx, slide) { return '<a href="#">'; }
  });
});

At first I thought it was a problem with my HTML code as I was using nested divs (by chance two internal divs per container div I wanted to cycle), but that wasn't the reason. Tracing the problem, I noticed, that application.js would get executed twice, and lo and behold, there it is at the top of the source: application.js included twice. Hmm, that's strange, my includes look like this:

<%= javascript_include_tag :defaults %>
<%= javascript_include_tag 'jquery.cycle.min' %>

In application.rb, I made the following configuration as suggested by the jquery_ujs github's README:

config.action_view.javascript_expansions[:defaults] = %w(jquery rails application)

As it turns out, however, rails is very smart about javascript files and when someone includes :defaults (and in :defaults only) it will automatically append application.js to the list of files to include. Let's take a look at actionpack/lib/action_view/helpers/asset_tag_helper.rb:

def expand_javascript_sources(sources, recursive = false)
  if sources.include?(:all)
    all_javascript_files = collect_asset_files(config.javascripts_dir, ('**' if recursive), '*.js')
    ((determine_source(:defaults, @@javascript_expansions).dup & all_javascript_files) +
      all_javascript_files).uniq
  else
    expanded_sources = sources.collect do |source|
      determine_source(source, @@javascript_expansions)
    end.flatten
    expanded_sources << "application" if sources.include?(:defaults) && File.exist?(File.join(config.javascripts_dir, "application.js"))
    expanded_sources
  end
end

Removing application from the javascript_expansions[:default] configuration removes the duplicate application.js and Cycle now works properly.

Yaml !!merge

dezso.zoltan@gmail.com (Zoltan Dezso) — Fri, 08 Oct 2010 13:07:58 -0700

There are two interesting constructs I came across in the database.yml in the typo source. One of them is for creating and using a named block, but the other one was a lot more interesting technique to grab parts of the Yaml tree and merge it into another part of the tree, which works in combination with the named blocks to result in a DRY configuration file like this:

  login: &login
    adapter: mysql
    host: localhost
    username: root
    password:

  development:
    database: typo_dev
    <<: *login

  test:
    database: typo_tests
    <<: *login

  production:
    database: typo
    <<: *login

OK, so the login: &login part creates a named block and *login simply uses that block. This is not incredibly difficult so far, but what on earth are those "<<:" ? Because I haven't seen them before, for a moment I was wondering if this is some sort of metaprogramming magic, where the << method/operator will somehow get called and execute with the block given to it. To find out how it works, I went to read the YAML 1.2 specification, but interestingly this syntax wasn't anywhere in the main doc. As it turns out, << is actually a shorthand for a tag, namely !!merge, and is mentioned in a separate document named _Language-Independent Types for YAML™ _

There are a few other tags mentioned here as well. In ruby, collection types seem to create a custom YAML::PrivateType (or Syck::PrivateType in 1.9) object and shove the data in its @value. I don't really see much use for these types in ruby now.

Scalar types on the other hand are better integrated, but tags like !!binary, !!value or !!yaml still create the PrivateType objects, so I guess they are not supported - and these are all only recommendations according to the document.

Testing Rails 3.0

dezso.zoltan@gmail.com (Zoltan Dezso) — Mon, 13 Sep 2010 16:26:14 -0700

Getting rails 3

Rails 3.0 was released this week (*), so I thought I'd go and try to run it's test suite - on windows.

The first step is to get Ruby 1.9.2p0 from the windows installer page and install it. I'm running on pik, so after installation and setup, I'd switch to 1.9.2 with

pik sw 192

The stock 1.9.2 install contains a slightly old version (2.5.8) of the rdoc gem, so the first thing to do is to update it with gem update.

Another tool that is necessary is the bundler gem, so install it with gem install bundler. Now to fetch all the prerequisites, all we need to do is run bundle install in the root of the rails source. (Note the command is bundle, the gem name is bundler - I sometimes forget this when typing fast)

Testing

What I wanted to do is to run the built-in test suite. Theoretically, just running rake test in the root should work and it would test every component, but in practice that doesn't quite work out that well (lots of warnings push the results out of screen pretty fast), so I opted to run the tests for each component separately.

activemodel

In the activemodel directory, entering rake test will run the test suite for the activemodel component only. It's result should be just a little bit messy with a lot of warnings, but in the end, we can get our first easy victory:

400 tests, 1110 assertions, 0 failures, 0 errors, 0 skips

activeresource

While this pack spews a huge number of warnings, the result is quite tame and we can more or less trust activeresource on windows.

272 tests, 789 assertions, 0 failures, 0 errors, 0 skips

actionmailer

Another easy victory in spite of some warnings.

175 tests, 479 assertions, 0 failures, 0 errors, 0 skips

activesupport

After running rake test for activesupport, the picture is slightly different, but still not unreasonable (these tests were run without memcached). What I found was

2055 tests, 9169 assertions, 35 failures, 1 errors, 0 skips

Let's see what these failures are. There are basically two areas of activesupport that do not pass on my windows machine. One is reasonable, the other one is slightly more difficult to understand: File operations (especially permissions) and Time. I've uploaded the exact output in this gist.

To dig a little deeper I looked at a few of the time errors.

3) Failure: test_current_returns_time_zone_today_when_zone_default_set
(DateExtCalculationsTest) [test/core_ext/date_ext_test.rb:397]:
<Sat, 01 Jan 2000> expected but was <Fri, 31 Dec 1999>.

The code that is responsible for the testing is the following:

def test_current_returns_time_zone_today_when_zone_default_set
  Time.zone_default = ActiveSupport::TimeZone['Eastern Time (US & Canada)']
  with_env_tz 'US/Central' do
    Time.stubs(:now).returns Time.local(1999, 12, 31, 23)
    assert_equal Date.new(1999, 12, 31), Date.today
    assert_equal Date.new(2000, 1, 1), Date.current
  end
ensure
  Time.zone_default = nil
end

Or in other words, 1999-12-31 23:00 in Central Time should be 2000-01-01 00:00 in Eastern Time. The difference between Date.today and Data.current is that Date.current is supposed to take the time zone into account.

def current
  ::Time.zone_default ? ::Time.zone.today : ::Date.today
end

So why doesn't it work on Windows? The answer lies in the with_env_tz method, which looks like this:

def with_env_tz(new_tz = 'US/Eastern')
  old_tz, ENV['TZ'] = ENV['TZ'], new_tz
  yield
ensure
  old_tz ? ENV['TZ'] = old_tz : ENV.delete('TZ')
end

This method sets the time zone to be used through an environment variable TZ. However on windows, this variable can only be set at script startup and not during runtime (its value will be ignored). In fact, this is the method that causes all the failing tests on windows. This shouldn't actually cause a problem for rails projects, though, the timezone is properly set up at startup.

A more interesting part is the exception raised in one of the tests. That one is coming from a test that tries to silence STDERR, which does not work on windows and raises Errno::EBADF, which is unfortunately not caught. A way to deal with it would be to catch this exception.

def test_silence_stderr
  old_stderr_position = STDERR.tell
  silence_stderr { STDERR.puts 'hello world' }
  assert_equal old_stderr_position, STDERR.tell
rescue Errno::ESPIPE, Errno::EBADF
  # Skip if we can't STDERR.tell
end

Moving on to the other filesystem-related errors, we can find that they are related to permissions. For instance the following test

def test_atomic_write_preserves_default_file_permissions
  contents = "Atomic Text"
  File.atomic_write(file_name, Dir.pwd) do |file|
    file.write(contents)
    assert !File.exist?(file_name)
  end
  assert File.exist?(file_name)
  assert_equal 0100666 ^ File.umask, file_mode
  assert_equal contents, File.read(file_name)
ensure
  File.unlink(file_name)
rescue
  nil
end

is testing if the file's permissions are kept as the default (0666 -rw-rw- rw-), however here's what the ruby documentations has to say about File.open()

Opens the file named by fileName according to aModeString (default is ``r'') and returns a new File object. See Table 22.5 on page 326 for a description of aModeString. The file mode may optionally be specified as a Fixnum by or-ing together the flags described in Table 22.3 on page 305. Optional permission bits may be given in aPermNum. These mode and permission bits are platform dependent; on Unix systems, see open(2) for details.

The general meaning of "platform dependent" seems to be "it doesn't work on Windows".

The default permission on Windows seems to be 0644, so in this case, the test itself could account for the differences in the platform and we could fix the test by writing something like the following:

DEFAULT_PERMISSIONS = Config::CONFIG[‘host_os’] =~ /mswin|mingw/ ? 0100644 : 0100666

Update: Thanks for Luis Lavena to point out the flaws in the original platform checking

Although this will not help with the tests that check if permissions can be set properly, at least it solves the default permission test. At this point there is only one final type of tests that fail, the FileStoreTests. It would probably be proper to skip these permission tests on windows altogether.

The remaining failures come from FileStoreTest in test/caching_test.rb:

test_crazy_key_characters(FileStoreTest) <3> expected but was <nil>.
test_decrement(FileStoreTest) <2> expected but was <nil>.
test_increment(FileStoreTest) <2> expected but was <nil>.
test_local_cache_of_decrement(FileStoreTest) <2> expected but was <3>.
test_local_cache_of_increment(FileStoreTest) <3> expected but was <2>.

It seems that actually all tests fail on the increment or decrement methods, so let's try to investigate which feature might be "platform dependent" this time.

def increment(name, amount = 1, options = nil)
  file_name = key_file_path(namespaced_key(name, options))
  lock_file(file_name) do
    options = merged_options(options)
    if num = read(name, options)
      num = num.to_i + amount
      write(name, num, options)
      num
    else
      nil
    end
  end
end

def lock_file(file_name, &block) # :nodoc:
  if File.exist?(file_name)
    File.open(file_name, 'r') do |f|
      begin
        f.flock File::LOCK_EX
        yield
      ensure
        f.flock File::LOCK_UN
      end
    end
  else
    yield
  end
end

As it can be guessed, the ruby documentation has this to say about File.flock: "Not available on all platforms." In fact the problem happens at read (or more precisely at read_entry) where a permission denied exception is raised, but is promptly swallowed and nil is returned instead. This makes increment and decrement to fail.

The bottom line about using activesupport on windows is that timezone-changing via environment variables will not work on the fly. Also, there is the problem with file-based caching that can easily lead to frustration, but otherwise activesupport can be trusted.

actionpack

Nice clean output for the most part and the result is also pretty nice:

2819 tests, 10698 assertions, 2 failures, 5 errors, 0 skips

1) Failure: test_caching_stylesheet_link_tag_when_caching_on(AssetTagHelperTest)
<55> expected but was <59>.

The failure is a difference in the response size and its expected size. That means that there is a 4 byte excess. And of course those bytes are coming from the CRLFs.

activerecord

Now, ActiveRecord's output is really messy on windows. In fact to the point that tests don't even run and all I'm left in the end is:

Errors running test_mysql, test_mysql2, test_sqlite3, test_postgresql

(*) And because this post already takes a lot longer than I wanted, I'll take a look at ActiveRecord in another post, and will conclude this one.

Referrer spam in statpress

dezso.zoltan@gmail.com (Zoltan Dezso) — Wed, 14 Jul 2010 16:50:40 -0700

Today I got really fed up with statpress referrer spam, so I decided to implement a dead simple referrer check function into statpress. The function works very similar to the current IP banning, so it wasn't very difficult to get it to work. If I get angry again, I might create a new statpress Option/Page to make the word banning easier or even make it non-destructive by only marking the referrer as spam. Of course other than the proactive banning, one might want to sanitize the referrrers in the statpress table as well. For me, I just cleared all the referrers with bad words in them, but didn't bother to entirely remove the entries as I wanted to keep count of the actual access count.

Here are my simple changes to statpress.php:

// A function to check whether a referrer should be
// omitted due to a manual ban
// To add/remove referrer bans, edit
// wp-content/plugins/statpress/def/banreferrers.dat
function iriCheckBanReferrer($arg)
{
  $lines = file(ABSPATH.'wp-content/plugins/'.dirname(plugin_basename(__FILE__)). '/def/banreferrers.dat');
  foreach($lines as $line_num => $banref)
  {
    if(strpos($arg,rtrim($banref,"n"))===FALSE)
      continue;
    return true; // banned
  }
  return false;
}

// In function iriStatAppend()
$referrer = (isset($_SERVER['HTTP_REFERER']) ? htmlentities($_SERVER['HTTP_REFERER']) : '');
// BANNED REFERRERS
if (iriCheckBanReferrer($referrer)) $referrer = '';
// END BANNED REFERRERS

Then all I needed to do is create a banreferrers.dat file in the plugins/statpress/def directory containing one bad word/domain per line and now hopefully a lot less referrer spam will appear in the statpress overview. For now this will suffice for me.

C for Ruby

dezso.zoltan@gmail.com (Zoltan Dezso) — Mon, 12 Jul 2010 17:34:46 -0700

Last weekend, I did an informal mini-conference on C and Ruby with a few friends.

Introduction to C

We started with an introduction to C as some of my friends didn't know it at all. Of course pointers in C are a little difficult to wrap heads around at first, but hopefully the things we covered will get them interested enough to try and follow up on them. One particular example that started many questions was something along these lines:

void change(int** i)
{
  *i = (int*)malloc(sizeof(int));
  **i = 123;
  // what happens if we do
  // free(*i);
  // here
}

int main()
{
  int *ptr = NULL;
  change(&ptr);
  printf("%d", *ptr);
  // we should use free(ptr); here (if main() wasn't exiting anyway)
}

Especially the part about free-ing before accessing the variable: what happens if we access the already free-d memory. On one machine we tried it, the number was still 123, which showed very well how much we cannot trust the compiler/OS to actually zero out the memory immediately when it is released and how easily this can lead to bugs that work sometimes and fail sometimes. Next we talked about the connections between C arrays and pointers.

int a[10];
int* b = a;

// an array is in fact a pointer to the head
a[0] = 1;
*a = 1; // this is the same as the above
a[1] = 1;
*(a+1) = 1;
// the C compiler knows how long an int is
// so the + 1 will point to the first element
// instead of just plus one byte

Next up using pointers/arrays, together we implemented a very easy function to check whether a string was a palindrom:

int test(char* str)
{
  char *lo = str + 0; // lo is set to the first character
  char *hi = str + strlen(str) - 1; // hi is set to the last character
  while (lo < hi)
  {
    if ((*lo) != (*hi))
      return 0; // the chars don't match
    lo++;
    hi--;
    // move lo to the right, hi to the left
  }
  return 1; // every char matched, it is a palindrom
}

Understanding Array#reverse

After we discussed the very basics of C, finally we started diving into the source code of Ruby (latest trunk from github). I wanted to keep it simple, so to be connected to the previous discussion, I left out the basics of VALUE and the first code we looked at was Array#reverse and Array#reverse! These functions were fairly easy to understand as there is no complicated resizing or in String#reverse's case, complicated encodings to deal with. In fact, before looking at the code, we discussed how it could be implemented - and to no surprise the actual implementation was very similar.

VALUE rb_ary_reverse(VALUE ary)
{
  VALUE *p1, *p2;
  VALUE tmp;

  rb_ary_modify(ary);
  if (RARRAY_LEN(ary) > 1)
  {
    p1 = RARRAY_PTR(ary); // get pointer to first item
    p2 = p1 + RARRAY_LEN(ary) - 1; // get pointer to last item

    while (p1 < p2) // imaging p1 as lo and p2 as hi
    {
      tmp = *p1; // swap *p1 with *p2
      *p1++ = *p2;
      *p2-- = tmp;
    }
  }
  return ary;
}

One thing to remember here is the *p1++ notation. To those unfamiliar with C, var++ means that first var is "returned" or in other words, var is used and only then is it incremented. (the ++ is after the variable). On the other hand, ++var would first increment var and then use this incremented value. So it is crucial to use *p1++ here instead of *++p1. Now in ruby, there are two versions of reverse, one with a bang and one without. From the code in the above helper function, we can assume that it is for the reverse! type, but it would not be very useful for Array#reverse as it would change the array in place. Instead, we saw in the sources that Array#reverse actually internally duplicates the array before doing the in-place reverse on the duplicated array. This pointed out that Array#reverse must be slower than Array#reverse! and of course it must also use more memory. Even though I didn't go into too much detail about Ruby's GC (which I'm not all that familiar with anyway beyond knowing it is a mark-and-sweep type collector, just like the CLR), I mentioned that the GC will be started when a memory allocation does not have enough memory. So in the end, not only is it slower and uses more memory, but there is a chance that Array#reverse triggers garbage collection which will make it even slower and use even more memory. (As an aside, realizations like these are the reasons I wanted to talk about the internals, as I believe understanding even just a little about how things work make us better programmers in the long term)

To understand the performance implications, we tried implementing Array#reverse! in ruby which we benchmarked against the native reverse!:

require 'rubygems'
require 'benchmark'

class Array
  def rb_reverse!
    lo,hi=0,self.length-1
    while lo < hi
      tmp = self[lo]
      self[lo] = self[hi]
      self[hi] = tmp
      lo += 1; hi -= 1
    end
  end
end

arr = (1..10_000_000).to_a
Benchmark.bm do |x|
  x.report { arr.reverse! }
  x.report { arr.reverse }
  x.report { arr.rb_reverse! }
end

The result was absolutely clear:

    user     system      total        real
0.030000   0.000000   0.030000 (  0.037438)
0.100000   0.040000   0.140000 (  0.142042)
6.290000   0.020000   6.310000 (  6.307183)

Extending Fixnum#succ

Next we looked at an (easy-looking) item from the Ruby ToDo file: * optional stepsize argument for succ()

By default Fixnum#succ doesn't take any arguments and just increments the variable by 1. So let's take a look at what it's doing internally:

static VALUE fix_succ(VALUE num)
{
  long i = FIX2LONG(num) + 1;
  return LONG2NUM(i);
}

No surprise here, it takes no arguments (besides a "this") and increments by 1. To fix this, I explained about variable argument list, so we can take one or zero arguments and naively implemented a first version of succ:

static VALUE fix_succ(int argc, VALUE* argv, VALUE num)
{
  VALUE step; // this will hold the value to increment by
  if (argc == 0) // in case we don't receive any arguments, use 1 as step
  {
    step = INT2FIX(1); // just like above
  }
  else
  {
    if (argc == 1) // when there is one argument, use it as the increment
    {
      step = argv[0];
    }
    else // if there are more than 1 arguments, raise error
    {
      rb_raise(rb_eArgError, "wrong number of arguments");
    }
  }
  long i = FIX2LONG(num) + FIX2LONG(step); // add the numbers and return
  return LONG2NUM(i);
}

Of course this in itself does not work, which gave me a chance to briefly mention how method lookup tables work. Then together we changed that table to indicate that succ might take any number of arguments with the entry:

rb_define_method(rb_cFixnum, "succ", fix_succ, -1);

However that naive implementation has a lot of problems that then gave me an opportunity to again mention other parts of ruby we should be aware of.

# For starters, let's try the new succ method:
1.succ
> 2 # OK
1.succ 2
> 3 # OK
1.succ 1073741823
> 1073741824 # OK
1.succ 1073741824
> 73693943 # Hey, what's going on here

# other errors
1.succ 'a'
> 73021957 # whoa, what's this?
'a'.succ
> b # OK
'a'.succ 2
> in `succ': wrong number of arguments(1 for 0) (ArgumentError) #Ooops

Of course to understand what the problem is, we should know why 1073741824 is special. As it turns out, Fixnum's largest representable value is 1073741823, which means that 1073741824 causes an overflow. Even though C int/long can go twice as high (2^31), Fixnum can hold up to 2³⁰ only. This means that values that are higher than 1073741823 will be of type Bignum, rather than Fixnum. So one thing we can do is to check if the argument is a Fixnum. Only if it is a Fixnum, should we add the values together, but when it's a Bignum, we can call into the regular + method, that will take care of everything for us. In fact, we could have directly implemented everything as an addition, but the point here is how to map C types to Ruby types and how we can deal with the differences and the limitations of the underlying language.

static VALUE fix_succ(int argc, VALUE* argv, VALUE num)
{
  VALUE step;
  if (argc == 0)
  {
    step = INT2FIX(1);
  }
  else
  {
    if (argc == 1)
    {
      step = argv[0];
    }
    else
    {
      rb_raise(rb_eArgError, "wrong number of arguments");
    }
  }
  if (FIXNUM_P(step)) // We can check if step can fit in Fixnum
  {
    // yes, just add it together in a long
    // remember, that long can be twice as big as fixnum,
    // so no worries about overflow
    long i = FIX2LONG(num) + FIX2LONG(step);
    return LONG2NUM(i);
  }
  // If it didn't fit into Fixnum, so let's just add it naturally
  else
  {
    return rb_funcall(num, '+', 1, step);
  }
}

This finally works for numbers, but still has some problems. Also, "'a'.succ 2" will not work, but the code for String#succ is very complicated because of the different encodings. As all my friends (except Huan) had to deal with emoji conversion at one point in their lives, it didn't take much to scare them away from looking at those codes (or even try to implement #succ - but they are free to do so if they want to)

Writing native extensions for Ruby

Finally I wanted to look at implementing Ruby extensions in C (and comparing performance) but unfortunately the mkmf lib was not found on my machine (Later I realized that it was because RVM changed to an unusable ruby version when I wasn't looking), so instead tried implementing binary search together as an exercise. As a note, here's the final working version:

def rbinsrc(ary, src)
  st,en = 0, ary.length-1
  while st <= en
    mid = st + (en - st) / 2 # avoid overflow in (st + en) / 2
    if (src < ary[mid])
      en = mid - 1
    elsif (src > ary[mid])
      st = mid + 1
    else
      return mid
    end
    return -1 # not found
  end
end

In fact, I created an extension to do the same thing and benchmarked the Ruby version against the C version

#include "ruby.h"
VALUE Convrt = Qnil;
void Init_convrt();
VALUE method_binsrc(VALUE self, VALUE arr, VALUE src);

void Init_convrt()
{
  Convrt = rb_define_module("Convrt");
  rb_define_method(Convrt, "binsrc", method_binsrc, 2);
}

VALUE method_binsrc(VALUE self, VALUE arr, VALUE src)
{
  int st, en, md; int s;
  if (TYPE(arr) != T_ARRAY || TYPE(src) != T_FIXNUM)
    rb_raise(rb_eTypeError, "Invalid arguments");

  st = 0;
  en = RARRAY(arr)->len-1;
  s = FIX2INT(src);
  while (st <= en)
  {
    md = st + (en-st)/2;
    if (s < FIX2INT(RARRAY(arr)->ptr[md]))
      en=md-1;
    else if (FIX2INT(RARRAY(arr)->ptr[md]) < s)
      st = md+1;
    else
      return (INT2FIX(md));
  }
  return (INT2FIX(-1));
}

require 'benchmark'
require 'convrt'
include Convrt

def rbinsrc(arr,src)
  st,en = 0,arr.length-1
  while (st <= en)
    md = st + (en-st)/2
    if (src < arr[md])
      en = md - 1
    elsif (arr[md] < src)
      st = md + 1
    else
      return md
    end
  end
  return -1
end

N = ARGV[0].to_i a = (1..100000).to_a

Benchmark.bm do |x|
  x.report { 1.upto(N) { binsrc(a,a.length-1)} }
  x.report { 1.upto(N) {rbinsrc(a,a.length-1)} }
end

And the result again is very clear, which shows how a C extension for Ruby can improve performance in bottleneck situations but because it complicates things a lot, how it should be only used with good judgment.

$ ruby test_02.rb 10000
      user     system      total        real
  0.010000   0.000000   0.010000 (  0.016662)
  0.430000   0.020000   0.450000 (  0.449629)

F# Discriminated Unions

dezso.zoltan@gmail.com (Zoltan Dezso) — Tue, 06 Jul 2010 00:50:53 -0700

In F# discriminated unions are providing a base for the powerful pattern matching. Let's examine what happens behind the scene when someone enters the following innocent-looking code:

type Test = | Option1 of float | Option2 of int

If we take the generated assembly apart, this simple code generates a relatively large amount of code. First of all, a class (type) named Test is going to be created. What is more surprising is that five nested classes will also be defined. Of this five, two seems to be crucial as we'll see while the others seem to be more about administration.

Talking about administration, the resulting type implements a fair amount of interfaces automatically.

IEquatable adds three Equals methods: one with the Test type, but also two with object type (one of which allows a second IEqualityComparer parameter). Similarly IComparable and IComparable are implemented. In addition IStructuralEquatable and IStructuralComparable are also implemented. I won't talk about these methods.

What is more important is that one nested class that is inherited from Test will be created for each option, named Option1 and Option2. Both of these classes will contain a property named Item and not surprisingly Item will be float64 in Option1 and int32 in Option2.

Test will also contain two more boolean properties: IsOption1 and IsOption2 which return whether this refers to an instance of the respective type.

Finally Test provides two factory constructors named NewOption1 and NewOption2 which create either of the nested option instances. Its constructor in fact is not public (and doesn't do anything except create the useless parent Test instance).

An implementation in C# therefore (removing much of the automatically generated interface implementations and helper methods) would look like this. The generated IL is very similar to that of the original F# code.

public class Test
{
  public class Option1 : Test
  {
    public Option1(double v) { item = v; }
    readonly double item;
    double Item { get { return item; } }
  }

  public class Option2 : Test
  {
    public Option2(int v) { item = v; }
    readonly int item;
    int Item { get { return item; } }
  }

  public sealed class Tags
  {
    public const int Option1 = 0;
    public const int Option2 = 1;
  }
  Test() {}

  public static Test NewObject1(float v) { return new Option1(v); }
  public static Test NewObject2(int v) { return new Option2(v); }

  public bool IsOption1 { get { return (this is Option1); } }
  public bool IsOption2 { get { return (this is Option2); } }
  public int Tag { get { return ((this is Option1) ? Tags::Option1 : Tags::Option2); } }
}

There are two further nested classes named Option1@DebugTypeProxy and Option2@DebugTypeProxy, but I'm not really sure what their functionality is. Also, there is that Tags nested class with its const definitions and the Tag property in Test whose role is also not completely clear from this IL code. It can be related to enums or maybe to do with performance

After comparing the two implementations, it is very easy to appreciate how - in probably Don Syme's favorite word - succinct F# really is (and that's even before diving into heavyweight functional programming.

F# Currying

dezso.zoltan@gmail.com (Zoltan Dezso) — Wed, 30 Jun 2010 22:00:32 -0700

Let's look at how currying works behind the scenes. It is not very surprising that this code

let adder x y = x + y
let partial x = adder 2 + y

is compiled to

.method public static int32 adder(int32 x, int32 y) cil managed 
{
  .custom instance void Microsoft.FSharp.Core.CompilationArgumentCountsAttribute::.ctor(int32[])
    = (01 00 02 00 00 00 01 00 00 00 01 00 00 00 00 00 ) // Code size 5 (0x5)
  .maxstack 4
  IL_0000: nop
  IL_0001: ldarg.0
  IL_0002: ldarg.1
  IL_0003: add
  IL_0004: ret
} // end of method Program::adder

.method public static int32 partial(int32 x) cil managed
{
  // Code size 9 (0x9)
  .maxstack 4
    IL_0000: nop
    IL_0001: ldc.i4.2
    IL_0002: ldarg.0
    IL_0003: call int32 Program::adder(int32,int32)
    IL_0008: ret
} // end of method Program::partial

which comes down to the following common C# code

public class Test { public static int adder(int x, int y) { return (x + y); }
public static int partial(int x) { return (adder(x, 2)); } }

and it does generate a similar IL code:

.method public hidebysig static int32 adder(int32 x,int32 y) cil managed
{
  // Code size 9 (0x9)
  .maxstack 2
  .locals init ([0] int32 CS$1$0000)
  IL_0000: nop
  IL_0001: ldarg.0
  IL_0002: ldarg.1
  IL_0003: add
  IL_0004: stloc.0
  IL_0005: br.s IL_0007
  IL_0007: ldloc.0
  IL_0008: ret
} // end of method Test::adder .method

public hidebysig static int32 partial(int32 x) cil managed
{
  // Code size 13 (0xd)
  .maxstack 2
  .locals init ([0] int32 CS$1$0000)
  IL_0000: nop
  IL_0001: ldarg.0
  IL_0002: ldc.i4.2
  IL_0003: call int32 CSharpTest.Test::adder(int32,int32)
  IL_0008: stloc.0
  IL_0009: br.s IL_000b
  IL_000b: ldloc.0
  IL_000c: ret
} // end of method Test::partial

F# Swap

dezso.zoltan@gmail.com (Zoltan Dezso) — Wed, 30 Jun 2010 02:20:46 -0700

Reading about functional programming, I got curious the other day: one of the canonical examples is swapping the elements in a tuple but what exactly happens internally? After all, whatever language is used, eventually it will be translated to native code.

F# is very easy to work with in this sense, because it will be compiled to IL first, so if I only run ILdasm on it, an equivalent C# code could be created.

let a(x,y) = (y,x)

This innocent short code will generate the following IL code

.method public static class System.Tuple`2<!!b,!!a> a<a,b>(!!a x,!!b y) cil managed
{
  // Code size 9 (0x9)
  .maxstack 4
  IL_0000: nop
  IL_0001: ldarg.1
  IL_0002: ldarg.0
  IL_0003: newobj instance void class System.Tuple`2<!!b,!!a>::.ctor(!0, !1)
  IL_0008: ret
}

Aha! So we are not fiddling around with temporary variables (which wouldn't work anyway because variables are immutable by default in F#), instead the simplest of things happens here: a new variable is constructed with the parameters swapped and then returned.

In C#, Swap could be written along these lines

public static Tuple<T2, T1> Swap<T1, T2>(T1 a0, T2 a1)
{
  return (new Tuple<T2, T1>(a1, a0));
}

In fact the C# code generates an almost identical IL code (although honestly I'm not sure what the final branch is supposed to be, as it doesn't do anything apparently useful)

.method public hidebysig static class System.Tuple`2<!!T2,!!T1> Swap<T1,T2>(!!T1 a0, !!T2 a1) cil managed
{
  // Code size 13 (0xd)
  .maxstack 3
  .locals init ([0] class System.Tuple`2<!!T2,!!T1> CS$1$0000)
  IL_0000: nop
  IL_0001: ldarg.1
  IL_0002: ldarg.0
  IL_0003: newobj instance void class System.Tuple`2<!!T2,!!T1>::.ctor(!0, !1)
  IL_0008: stloc.0
  IL_0009: br.s IL_000b
  IL_000b: ldloc.0
  IL_000c: ret
}

OSGrid in Rei

dezso.zoltan@gmail.com (Zoltan Dezso) — Mon, 21 Dec 2009 19:26:02 -0800

Textures

Before Rei was released at the end of September, it was called 3Di OpenViewer. That version didn't need to download textures through the region server, rather, it would download assets (textures, meshes, avatars) directly from dedicated asset servers. For this reason, when someone logged in to an upstream OpenSim grid with 3Di OpenViewer, textures and avatars would not appear.

But that changed when we released Rei as an open-source viewer at the end of September. At that time, textures were already being fetched from OpenSim. You can go ahead and grab the installer from github and it will display textures. What it won't do with textures is correctly apply UV coordinates in some (fortunately very rare) twisted-hollow-prim cases (but it was more about PrimMesher than Rei). Since September, we have done many improvements to PrimMesher - and Rei too, so now even those rare cases appear correctly (big thanks to the bug reporters). Texture downloads are slow unfortunately, but more on that later.

Avatars

Now this is a very delicate issue. The heart of the issue is that currently there is no non-LL viewer that supports SecondLife™ avatars. Let me say that again: there are no viewers that support SL™ avatars. Naali has a built-in mesh (the same as their old viewer), IdealistViewer has a built-in mesh, LookingGlass doesn't yet have avatars (correct me if it does), Xenki doesn't have an avatar yet as far as I know either.

The closest a viewer gets to SL avatar support is the PandaViewer by dahlia (http://vimeo.com/6812213) , which supports avatar shapes and actually looks really nice. Unfortunately it's not open source (yet?), nor is it available in any form.

The thing with avatar support is that it's pretty difficult to do right. No, not only technically, but there's the issue of using or not using Linden Labs™'s original mesh. You can find downloadable meshes, but there is a nice warning hidden neatly inside an obj file:

This model is released for use by residents of Second Life. It may be imported into external 3D modeling, texturing, and animation applications to aid in the creation of character animations and textures for Second Life, as well as for any non-commercial purpose. Use or modification of this model for any other commercial purpose is prohibited without the express written consent of Linden Lab.

With all this said, we are looking at SL™ avatar support. It definitely won't happen before the end of this year, the target is early next year.

OpenSim support

Of course Rei still has issues with upstream OpenSim support. This is partially due to the very very bad way textures are downloaded (through a single server, in UDP). The truth is, even the SL™ viewer chokes on those: in heavy regions it would download the low resolution image, but the full image is never actually downloaded. My personal opinion is that the sooner we can break free from that way of transferring assets to clients, the better (guess how 3Di OpenViewer delivers content, through dedicated asset servers). Hopefully OpenSim will not remain an SL™ clone and there will be lots of innovations in the virtual world space coming from OpenSim. Rei is supposed to show what is possible when we can control both the viewer and the server side without the fear of not being able to work on one or the other for six months.

The current state

LBSA Plaza

Wright Plaza

So, what do you think of the viewer so far? What sort of improvements do you think would be nice? What do you think is missing, other than the avatar support?

In general, what are the things you absolutely expect from a virtual world viewer?

Let me know in the comments.

Wii Controller Plugin for 3Di Viewer Rei

dezso.zoltan@gmail.com (Zoltan Dezso) — Fri, 20 Nov 2009 17:56:53 -0800

Wii Controller Plugin

3Di Viewer "Rei" provides a great flexibility when it comes to adding custom functionality to the core viewer by making it possible to create binary plugins. Just drop these binaries into the Rei folder and let the viewer find and load it automatically the next time it starts. While these plugins are still in their early phase - we can do a lot more to ease the process of deploying plugins and resolving their dependencies - you can get a glimpse at their power already.

The WiiControllerPlugin is a fun project, a proof-of-concept, rather than a commercial solution, so before you dive in, be warned, that it is sometimes rough around the edges. With that out of the way, if you have Rei, head over to http://github.com/zaki/ReiWiiControllerPlugin and grab the binary from the Download section. Just copy the contents into the Rei installation folder (usually C:Program Files3DiRei). And... that's basically it.

How to connect the controller?

To use a Wiimote, you will need a Wiimote, a nunchuck and a bluetooth adapter. I've heard that Toshiba adapters works best, but I have used Broadcomm's chips successfully as well. It is said that the Microsoft stack doesn't work most of the time with the Wiimote.

Before you start the viewer, connect/pair the Wiimote via bluetooth. If you find it difficult to connect, I found that keeping (1) and (2) pressed simultaneously while connecting helps a lot.

Once the Wiimote is connected, start Rei and if everything goes well, the primary Wiimote will have its first LED turned on. You are now ready.

How to use the Wiimote?

The WiiControllerPlugin allows you to use the Wiimote and the nunchuck to control your avatar and the camera. The nunchuck's joystick, or the directional controller on the wiimote can be used to make you avatar walk. Up will always make your avatar walk forward, regardless of the position of the camera.

The camera is controlled with a combination of the wiimote and the nunchuck. Zoom in and out by holding down (Z) and tilting the nunchuck forward and backward. Hold the wiimote straight up; while you press (B), you can tilt the wiimote in four directions to rotate the camera around your avatar. Remember, you don't have to look away from the screen to change the camera angle.

If you get lost with the camera, you can always press (C) to go back to behind your avatar. Pressing (C) also switches to "follow" mode, so your camera will "stick" to your avatar and will follow it from behind if you move. Pressing (C) another time will switch back to the free camera mode.

If you get tired of moving with the wiimote and want to switch back to using the keyboard or mouse, you can disable avatar movements by pressing (-). Your wiimote's will light the third LED when it's disabled. Press (-) again to enable moving by the wii controller.

About the demo

The video above contains a very dull maze, in which there are colored balls that hurt or heal you when you touch them (I've never said I was a great contents designer, have I :)) The demo shows how easy it is to integrate custom features with an unmodified OpenSim grid. All that was used is simple LSL script that detects collisions and gives commands via llSay. These commands can be intercepted by javascript and further dispatched to all plugins, that can act accordingly. In this case a simple health bar is adjusted.

As my LSL was sending plaintext commands like "heal,20", the necessary javascript looks really simple:

function DispatchCommand(message)
{
  var strList = new Array();
  strList = message.split(",");

  if (strList.length == 2)
  {
    ctrl.Callback(strList[0], strList[1]);
  }
}

If you have a question, or have a good idea about plugins, or the WiiControllerPlugin, please share it in the comments.

Please note: WiiControllerPlugin is not endorsed by Nintendo. WiiControllerPlugin depends on WiiMoteLib, which is release under the license MS-PL. For more details, see http://wiimotelib.codeplex.com/

Plugins for Rei

dezso.zoltan@gmail.com (Zoltan Dezso) — Wed, 11 Nov 2009 18:09:23 -0800

Plugins

In “Rei”, there is a curious little green “health” bar above the avatar’s head. That bar is not something that is built in to the viewer, rather, a simple plugin provides the functions that are necessary to have a very rudimentary health bar. Of course this TestPlugin is only to show what sort of things are possible with “Rei”.

HEALTH BAR
DAMAGE
HEAL
DEAD

What do plugins do?

Plugins are intended for two main purposes. First, to communicate - through javascript - with the 2D web, and second, to interact with objects within the 3d scene. Of course a plugin is a .NET assembly, so it can pretty much do anything, but the main functions it is intended for are the above two. For instance TestPlugin provides two javascript functions: one to inflict damage and one to heal, basically allowing a regular 2d web application, for instance an SNS site or an online game, to change the health meter of the avatar within the 3d world according its own rules. Responding to these javascript functions then, the plugin interacts with the 3d world by rendering its own little health bar above the user’s head.

How to write plugins?

To make things as simple as possible, “Rei” uses the same method for developing plugins as OpenSim. Essentially one writes an addin for the /OpenViewer/Managers extension by implementing the IManagerPlugin interface and then creating and embedding a plugin manifest to the assembly.

IManagerPlugin

All plugins need to implement IManagerPlugin, which means implementing a few methods:

void Initialise(Viewer viewer);

- Initialise is the method that will be called right after constructing the plugin class. This will be only called once for the life of the plugin and is used to create objects that will persist through the entire lifetime such as static resources (textures, models), configurations.

void Initialize();

Initialize (note the z in the name) will be called when “Rei” is getting ready to enter a region. This happens on the first startup as well as after logout or before entering a new region after a teleport. This is to make sure that unnecessary scene objects do not make it over to a new environment, therefore Initialize would create objects that persist only through the lifetime of one region, such as dynamic resources (textures, meshes) or dynamic information (region data, prim data)
Liquid error: Flag value is invalid: -O ""

- Update is called every frame. This is the place to calculate the next state of the plugin, including objects in the 3d scene. This is the only place where it is allowed to change properties of any object that is within the scenegraph. By using the frame parameter, it is possible to count or skip frames.

void Draw();

Draw is also called every frame and it is where drawing directly to the 3d canvas is possible. Generally it is recommended to use proper scenenodes that draw automatically, instead of drawing directly.

void Cleanup();

Just like Initialize is called before “Rei” is ready to log in to a new region, Cleanup is called every time a region is no longer needed, such as when logging out or when leaving a region by teleport. Cleanup is responsible of releasing resources that were allocated by Initialize (but not Initialise!)

RefController Reference { get; set; }

This property is the entry point to "Rei"s core. Through this object it is possible to call into the public api of the viewer or to access javascript. Reference must always be initialized from the viewer parameter passed in to Initialise()

Communication with Javascript

Plugin to Javascript: Dispatch

A plugin can call Reference.Viewer.Adapter.Dispatch(string action, string message) when it wants to send a message to javascript. If a handler was added to the javascript library for the particular action, this dispatch will be processed by passing the message to the handler function. For future compatibility it is recommended although not required that the message be passed in json format.

Javascript to Plugin: Callback

The Adapter supplies a Callback delegate and a registry to handle calls from Javascript. A plugin can register a callback method that it is interested in receiving by calling

Reference.Viewer.Adapter.RegisterCallback

. It must pass the name of the method it wants to listen to and the method to call when the event arrives. The way a Callback works is very much like delegates work in the .NET framework.

From the javascript side, one can use

ctrl.Callback("method", parameters)

and all plugins that are listening to “method” will receive a callback to their registered functions. For an example of Callbacks, you can see the DoDamage and DoHeal from test.js and TestPlugin.cs

Plugin to Plugin: Message

It is also possible to send messages from one plugin to another, even without being aware of the other plugin. The mechanism is very similar to Callbacks: a plugin registers a message handler by calling

Reference.Viewer.Adapter.RegisterMessage

. The first parameter must be the name of the message to listen to and the second one is a message handler delegate.

A plugin can then raise a Message by calling

Rerence.Viewer.Adapter.SendMessage("message", parameters)

and all plugins that subscribed to the message being sent will receive it.

The contents of this post also appear on the github wiki for 3Di Viewer "Rei". For more up-to-date information, refer to REI PLUGINS WIKI

Irrlicht.NETCP and 3Di Rei on Linux

dezso.zoltan@gmail.com (Zoltan Dezso) — Thu, 05 Nov 2009 12:49:06 -0800

I have started a cleaner code line which is now rebased to Irrlicht 1.7.1. Find the codes on Github:Irrlicht.NET

Irrlicht.NETCP

I've spent the last couple of weeks working towards rebasing the now abandoned Irrlicht.NETCP to use the latest Irrlicht released: 1.6. There are multiple reasons to do so, probably the most straightforward is to get the bugfixes or some of the new features, like the new light manager. However the less direct reason to do the rebase is that it includes Mac support out of the box - and now that I have a macbook sitting on my desk at work (albeit, temporarily), that is something of very high interest for me. In my dreamland, OpenGL ES support (for the iPhone) would work without much work too - hey, I can dream, can't I.

So, for this rebase, I've started a new repo on github, that I hope will soon become the official repository for 3Di Viewer "Rei", and maybe even for users of Irrlicht.NETCP (if any are still left, but who knows, maybe this can spark some new interest). The repository starts with a clean Irrlicht 1.6 SDK and gradually adds the fixes and features that are necessary for Rei (but of course those are fixes and features that would probably come up in other development as well).

On top of the improvements to Irrlicht, there is one huge patch, that makes the IrrlichtW wrapper compatible with 1.6. This mostly consists of type changes in the method interfaces and other minor details, but the patch itself is pretty monstrous, as many things changed at the same time. It is only after this collective patch, that the .NETCP library is usable again (it won't even compile before this fix, so be careful with the history).

Later, IrrlichtML fixes were added. ML stands for multilingual and these changes are required even though things finally started moving away from c8* to proper strings and io::paths in the core, stuff like IME support was not included and the unicode support is still horrible to say the least.

Getting Irrlicht.NETCP

If you want to give the new Irrlicht.NETCP libraries a try, head to github: http://github.com/zaki/irrlicht.netcp and check out the sources (either using git, or click on download in github). For the impatient, I've also uploaded binary files for 32-bit windows (Release mode).

To compile the libraries yourself, you will have to compile three components: Irrlicht SDK, IrrlichtW and Irrlicht.NET, in basically this order. To compile Irrlicht SDK, go to Irrlicht SDKsourceIrrlicht and open Irrlicht9.0.sln in Visual Studio 2008 and build. IrrlichtW's solution is found in IrrlichtW, named IrrlichtW.sln

Irrlicht.NET uses Prebuild to create its solution files, so before you can compile, the file runprebuild2008.bat needs to be run. That creates the solution file Irrlicht.NET.sln, that can be opened and built in VS.

Rei and Irrlicht.NETCP

To use the new binaries with "Rei", some minor changes need to be made to Rei's source. I committed these changes to my github repository: http://github.com/zaki/3di-viewer-rei, however the changes are not in the master branch, which I try to keep as a clean integration copy (therefore there is a little time delay). Get the Irrlicht 1.6 supporting sources from the 'linux' branch.

One more thing...

Ahem... did I just say 'linux' branch? Well, that's probably because I've also started working on Rei linux support and so far things look quite bright (as opposed to the MacOSX support, but maybe later about that). I managed to make Rei run on my Ubuntu machine in standalone mode using Irrlicht.NETCP and mono. There are still a ton of things that need fixing, but it's only a matter of time before a consistent framerate is achieved - I had driver issues and occasional performance problems, probably not unrelated. There are some problems with JPEG2000 support that I didn't have time to look at yet and haven't written shaders (OpenGL seems to hate HLSL shaders for some reason :), but everything in its own time.

So if you feel particularly adventurous, go ahead and grab the sources from the 'linux' branch - Irrlicht.NETCP's master branch now includes linux support -, compile them on your linux machine and do let me know how it works for you.

Intelligent evolution

dezso.zoltan@gmail.com (Zoltan Dezso) — Wed, 16 Sep 2009 04:26:43 -0700

The two publications appeared practically at the same time and they started an argument of epic proportions within the scientific community. Both scientists claimed nothing less than having solved the mystery of the origin of life - and not less important - of humankind. Two distinctly different, even contradicting theories, two proofs. Both of them offering a complete solution: from the proposed axioms, both of them explained the diversity - the development of the individual and the similarities between species not only those currently existing but even those long extinct.

Some questions did remain unanswered though. The two schools differed greatly on their views on the mass extinctions during which time whole families of species have disappeared leaving only a faint trace in the form of fossil evidence: a skeleton here and there or an object we could connect to ancient - primitive - species. The community was simply amazed when remains of a smaller colony from a couple of thousands of years ago was unearthed, but even that could not compare to what excitement spread through scientific journals when fossils of a long disappeared ancient species were found in a rare excavation. Scientists from both camps would throw themselves at the new samples, trying to fit the new pieces of the puzzle to their theories.

One group has claimed, that species evolved through a very coordinated, but seemingly random process from other - usually more primitive - species. Through time, organs that were more useful for survival have been - in a manner - selected. It's as if there was an algorithm in nature that experiments with randomly changing the variables, keeping results that it deems good while terminating those it finds unworthy. This algorithm has been present for millions if not billions of years (some even went as far as proposing that the same process can be found before life itself appeared). There was however a strange gap between humans of the long gone past and humans of the present - as if only the right fossils that would bridge this gap were hiding somewhere, waiting to be found one day.

Followers of this theory - called evolutionists - found that pieces of evidence fit their theory perfectly. They argued, that evolution of the present humans from ancient human forms is completely possible, as the natural selection process would experiment practically randomly, therefore given enough time, ancestor species could be significantly different from descendant species. The other group also proposed a complete explanation, however their theory immediately caused an uproar in the scientific community. They claimed that present form humans have not appeared through a lengthy process of evolution, rather, it was created through an intelligent design. Their theory proposed that present day humans were created by an ancient, intelligent entity, or even by the - in lack of a better word - consciousness of some ancient species. Many would instantly distance themselves from members of this second group and some even went as far as calling the professor that first published this theory an idiot. As the arguments started to spiral out of hand, finally a well-known scientific association has organized a public forum for members of these groups where they were given the chance to present their findings and evidence. In the heated debate, followers of the school of evolution would claim that intelligent design is unscientific, while proponents of intelligent design would argue on the improbability of random evolution. A noted professor of the second group was quoted to say:

"I firmly believe that humans were created by an ancient intelligent designer. The probability of complex organs evolving at random is not in accordance with nature. The probability of a complex structure evolving randomly is provably smaller than the theoretically conceivable smallest probability in the Universe. It is laughable to assume that the organ we know as our brains have evolved at random from lesser life forms. I don't think it is even theoretically possible that the our alleged ancestors' primitive carbohydrate-based organs would spontaneously evolve into our immeasurably complex silicon chips."

Slowly getting there...

dezso.zoltan@gmail.com (Zoltan Dezso) — Tue, 08 Sep 2009 01:20:37 -0700

OSGRID IN 3DI OPENVIEWER

OSGRID IN HIPPO VIEWER

Pretty close, but still not close enough. I guess, I'll need to take a deep, hard look at PrimMesher too.

How to install and use 3Di OpenViewer

dezso.zoltan@gmail.com (Zoltan Dezso) — Wed, 19 Aug 2009 02:06:27 -0700

For updates about 3Di Viewer "Rei"'s features (particularly textures and avatars), also see the post OsGrid in Rei

How to install 3Di OpenViewer?

First, please note, that the viewer is currently for Windows only and 3Di only supports Internet Explorer 6+ and Firefox 3.0 and 3.5 at this time.

Step 1.

Internet Explorer

Go to http://www.3di-opensim.com/openviewer/ie_install.html

The ActiveX bar will appear, click Install

Firefox

Go to http://3di-opensim.com/openviewer/ff_install.html

Click on the orange button with the arrow to download and install the plugin.

Step 2.

The installer will appear.

The first screen is only an introduction. It asks you to close other applications before clicking next. The left button says Next, the right one is Cancel.

The second screen is the EULA. I won't try to translate it to english, you can get a raw English version by pushing this through any online translation service. The long and the short of it was summarized in these two sentences:

No commercial use
No reverse engineering

The first radio button says Agree, the second says Disagree. Again the buttons are Back, Next and Cancel in that order. I guess it is pretty difficult to Agree with something you can't read (I always feel that way in Japan :) ). There are no traps in there and as long as you don't use it to become a millionaire celebrity without 3Di's consent, you should be alright. (If you happen to be a millionaire celebrity already, you should definitely contact us :) )

The next screen has three buttons saying Back, Install and Cancel respectively. After clicking Install, it will start the automatic install process. When it finishes, you can click on the Finish button on the next screen.

Is it possible to use 3Di OpenViewer in English?

This question is asked many times lately, so here's a short guide on how to do it. Please do note, that this is sort of a pre-release thing, that might in some places be a little rough around the edges. And if you find any leftover Japanese text after the switch, do let me know.

When you install the viewer, and open a page with the viewer embedded, it will appear in default Japanese locale, but it is possible to change this to English. There are two ways to do it: you can log in to an OpenSim grid with the Japanese version and change the locale from the menu, or you could change the configuration file to make the viewer appear in English from the first login page. Either method you choose, you will have to start the viewer at least once (to initialize the necessary directories and configuration files).

Method 1 - Using the menu

Using the login screen, specify the login URI, first name, last name and password in that order, then click on the button to log in.

After you log in, move the cursor towards the upper right corner until the menu appears. Click on the wrench icon to go to settings. The last selectbox on the first tab is the locale, you should select the first option and press OK. A dialog will appear telling you that the settings will be effective after you restart the viewer. Do so, and the UI will appear in English now.

Method 2 - Changing the configuration file

After you start the viewer, the configuration file will be created in your local settings directory. On XP, this will be in Documents and Settings/<username>/Local Settings/Application Data/3Di/OpenViewer/, while on Vista and Win7, this will be in Users/<username>/AppData/LocalLow/3Di/OpenViewer

Open the file in configs/ called OpenViewer.ini and change locale to en

locale = en

Start the viewer again and it should be in English now.

Can I connect to OpenSim grids?

The short answer is 'yes'. The long answer is 'yes, but'. You can definitely log in to OpenSim grids with the current release version and prims will display. However textures and the avatar will not currently appear. The reason for this is that assets are not downloaded through the region server. We are working on a fully compatible version that we will release in the future, but for the moment to get the full experience, unfortunately you will have to connect to 3Di's servers. We are also preparing more demo servers, but I have to ask for just a little more patience on that.

If you try 3Di OpenViewer, your feedback is always appreciated.

FreeSWITCH, OpenSim and the G722.1C codec

dezso.zoltan@gmail.com (Zoltan Dezso) — Mon, 18 May 2009 17:48:53 -0700

Kai Ludwig has recently left a very valuable comment (thank you, Kai):

Actually the new FreeSwitch siren14 codec has never been used with SLVoice! Everybody having voicechat with OpenSim is using PCMU without knowing it. Kai Ludvig

I didn't realize that at first either, but after he pointed this out, I started looking for a way to enable siren14. I haven't really come up with a way to use the SLVoice.exe included with the SecondLife viewer, rather, I modified the voipforvw codes to support it. Voice quality is now way better than the stock PCMU.

How to compile Voipforvw's SLVoice

dezso.zoltan@gmail.com (Zoltan Dezso) — Sun, 10 May 2009 01:18:08 -0700

What is Voipforvw?

With the SecondLife viewer comes an executable named SLVoice.exe. This is the piece of software that is responsible for the "low level" voice communication, so while the viewer does the accounting (login information, UI), this part does all the heavy lifting of contacting the voice servers and of course translating the voice of the speaker to RTP packets and back.

voipforvw (voip for virtual worlds) is a replacement for this executable. The core of this application was originally developped by 3Di

Why should I care about it?

The original SLVoice was developped for Linden Labs by a company named Vivox and has a proprietary license. The good news is, that it is insanely high quality (it runs on a couple of platforms and powers virtual worlds like SecondLife and games like EVE Online). It also has 3D positional audio.

The bad news is that it's proprietary to the point that you are explicitly disallowed from using it for OpenSim. The FAQ on vivox's home page explicitly says:

Can I distribute this to friends?
No, currently Vivox software and service is offered for individual, non-commercial use only.

Can I use this outside of SecondLife
No, the currently offered software is only available for the Second Life Viewer open source community.

What's the catch?

Well, unfortunately there is a catch. The license of the SIP/RTP stack used in vopforvw is GPL. This has a simple effect on the whole project, that is, voipforvw automatically becomes GPL. Unfortunately currently there is no good library that comes with a BSD-like license, the closest we can get is LGPL. (If you happen to know of one, please do let me know in the comments). While I'm a fan of C#, there is basically no free SIP/SDP library of any license for C# either.

Second, at the moment noone is maintaining the Linux version: in theory it should compile just OK, but that is not something I'd personally guarantee. I will try to look into that later.

But it's difficult to use.

I've heard this argument for quite some time: it is probably my failure to not do much about it . Compiling the SLVoice replacement is not really that difficult. If you have problems with building, take a look at the screencast below and see if it can help fixing your problem. And of course, you can leave comments here

Please note: trunk contains a version that only works with SLViewer versions below 1.22 (1.19, 1.20 and 1.21 were tested). For 1.22, I started a branch named sl_1.22 that contains an updated version that - in turn - only works with 1.22. As I am not allowed to look at the viewer's source code, many thanks go to yk, who described the major API changes to me. Unfortunately this also means, that I can't guarantee that the 1.22 version works properly. At least I had success making conferences work with that branch, but further testing will be needed for it to be integrated back into trunk.

What will I need?

You will need voipforvw, that you can check out from sourceforge.net:

svn co https://voipforvw.svn.sourceforge.net/svnroot/voipforvw voipforvw

Next, you will need boost, curl and pjsip. The versions that I was using are 1.39, 7.19.4 and 1.0.2 respectively. For PjSIP, 1.x series will currently not work due to API changes.

For compiling on Windows, you will also need the Platform SDK and DirectX SDK. Voipforvw uses CMake for its configuration too.

Step 1. Compile dependencies

Set up the platform SDK and DirectX SDK folders in Visual Studio.

Compile boost with bjam. To build bjam first, just run build.bat, which creates bjam.exe; use that in the boost directory with the following commandline:

$ bjam --toolset=msvc link=static runtime-link=static threading=multi --with-thread stage

To compile cURL, open the solution file, change the build type to Release and set the Runtime Library to Multithreaded (/MT). These changes are necessary, because we will be using multiple boost state machines from a multithreaded environment. Build only libcurl.

PjSIP will need you to move the file pjlib/include/pj/config_site_sample.h into config_site.h No changes are necessary to this file, the defaults will work fine.

Open pjproject-vs8.sln (and convert it as needed) set it to Release and build the solution.

Step 2: Configure and build voipforvw

Next, go to the voipforvw trunk and edit CMakeLists.txt You will need to change the paths for PJDIR, CURLDIR and BOOSTDIR. Don't use backslashes, replace them with forward slashes. The CURLLIBS variable needs to be changed from curllib.lib to libcurl.lib.

Open up CMake-gui and set the source directory to the voipforvw trunk. The output directory can be the same. Hit Configure (it will probably complain about backwards compatibility, but just ignore it and hit Configure again). If configuring goes well, there will be no red lines in the config window. Hit Generate, which will ask for the format of the solution file, choose the Visual Studio version you have and click OK. This creates the main solution file.

After opening this solution, there are three things you will need to check: first, set the build type to Release. Next check that the Runtime Library is Multithreaded (/MT) to match the other libraries and finally, LIBCMT will have to be removed from the libraries (because it conflicts with MSVCRT). Now you can build SLVoice.exe that you can use as a replacement for the vivox solution.

Debug

You probably noticed that we are building everything in Release mode. That's fine as long as it works, but sometimes you will want to debug (hopefully less often). In this case you will need to change the Runtime Library to /MTd for PjSIP and voipforvw. Then CMakeLists.txt needs to be updated to reflect this by setting

SET (PJTARGET i386-win32-vc8-debug)

Where you remove the LIBCMT library, now you will have to do the same thing for both LIBCMT and LIBCMTD and you can build SLVoice.exe in debug mode.

If you only need logging to see what's going on inside, you can change the code:

// CHANGE FILE: main.h:37
#define LOG
#ifdef LOG
extern FILE *logfp;
#define VFVW_LOGINIT()     logfp = fopen("test.log", "a")
#define VFVW_LOG(fmt, ...) fprintf(logfp, "[VFVW] %s:%d - ", __FILE__,__LINE__); \
                           fprintf(logfp, fmt, ## __VA_ARGS__); \
                           fprintf(logfp, "\r\n"); \
                           fflush(logfp)
#else
#define VFVW_LOGINIT()     void()
#define VFVW_LOG(fmt, ...) void()
#endif

//CHANGE FILE main.cpp:15
#ifdef LOG
FILE *logfp = NULL;
#endif

3Di OpenViewer Feature Focus: Login

dezso.zoltan@gmail.com (Zoltan Dezso) — Fri, 01 May 2009 16:04:03 -0700

3Di OpenViewer provides a number of ways to connect to a grid, let's look at those in a little more detail.

Manual

Manual login is the normal way, that everyone using SecondLife is already used to. This mode will show a login box right within the application and will allow you to specify the login URI, the first and last name and password. You can set the default values with additional params as well.

  <param name="LoginMode" value="manual" /> <!-- Optional-->
  <param name="ServerURI" value="http://1.2.3.4:10001" />
  <param name="FirstName" value="first" />
  <param name="LastName" value="last" />
  <param name="Password" value="password" />

Click

Click login is great if you want to specify all the information for the login and not allow the user to change it in any way (for example dynamically creating the login page based on already existing session information). In this case, the user can only see an empty screen with no login box; when they click on this screen, login starts automatically.

Of course in click mode, you will have to specify all the login information like above and possibly provide explanation somewhere on your site.

 <param name="LoginMode" value="click" />

Hide

Users can also use a JavaScript function to log in. Sometimes, it is better not to allow users to specify any information, but at the same time, login information can dynamically change at any time. In this case, you can set the login mode to hide, which will not show the login box, but also not allow login from within the application. Rather, it only allows external JavaScript logins.

 <param name="LoginMode" value="hide" />

To log in from JavaScript, use the following function:

openviewerCtl.Login(firstname, lastname, password, server_uri, login_location);

For the JavaScript login method, it is also possible to specify the exact location to log in to. The format is

// " uri:regionname&x&y&z"
     uri:region1&128&128&25

FreeSWITCH module in OpenSim

dezso.zoltan@gmail.com (Zoltan Dezso) — Tue, 28 Apr 2009 03:26:01 -0700

Update: OpenSim has changed substantially since this post, so my explanation will probably not work anymore.

It seems that Linden Labs has finally removed the hardcoded vivox.com references in 1.22 making third party voice chat work with the original slvoice.exe.

At the same time Rob Smart from IBM stepped up the game and provided a solution for FreeSWITCH that now also works without an SLVoice replacement. I've also been looking at FreeSWITCH for a while and just when I finally have some spare time to play around with it, this new module comes out: and it's really awesome! First of all, it works without having to install kernel modules or suffer the lack of proper SIP Session-Timer (RFC4028) implementations.

There are three components needed for this setup.

OpenSim

Because the new FreeSWITCH module is experimental, it is only found in trunk, and as usual, please note, that trunk is unstable it might have serious bugs, it can corrupt your data, or eat your children, especially if you do the unthinkable and try to install it in a production environment, which you should never do. Ever! No exceptions. If you are brave enough, you can proceed very carefully and try out the trunk.

FreeSWITCH

I tried the Windows binary snapshot from here: and it works well. For configuration instructions take a look at the OpenSim wiki page. Linux installation instructions are also included there.

SecondLife Viewer

Only versions later than 1.22 are supported, so be sure to grab a newer version from Linden Labs.

Final touches

After setting up FreeSWITCH and OpenSim, connecting to the grid and enabling voice chat by default nothing will happen. Conference chat won't work with the default estate settings, because parcel voice is disabled. In the video, I'm setting "Use a private spatial channel", but "Use the estate spatial channel" should work as well. Enable voice, and it will work now. After this, conference and private chat features will work. On my setup, I found that private chat is quite clear and easy to understand, but the spatial conferences are way too low quality, but that might be a configuration issue on my environment.

3Di OpenViewer

dezso.zoltan@gmail.com (Zoltan Dezso) — Fri, 24 Apr 2009 03:31:19 -0700

Disclaimer: I am a senior engineer working on multiple products - including various OpenSim technologies and most recently 3Di OpenViewer - at Japan-based 3Di Inc. This is a personal blog, and my opinion does not necessarily always represent that of my employer.

Introduction

3Di has released (Japanese) its virtual world viewer last week. It is an embedded viewer, that works directly in a web browser; it connects to the virtual world server solution also provided by 3Di, called 3Di OpenSim Enterprise 1.0 (Japanese) that has quite a few addition on top of the community version, utilizes much more than just SL prims to create a nice looking environment even within the limitations of the web-browser.

Open?!

Let's get the biggest question out of the way first: the virtual ink on the press release hasn't even dried, but I was already asked where one could download the source code. After all, it is called _Open_Viewer isn't it.

The short answer is: it will be released as open source at a later - not too distant - stage. Personally I think that opening up the viewer will be crucial for two simple reasons: first, the future of OpenSim depends on a viewer that opensim developers can look at without running the risk of getting burned by the license - and serious innovation requires some degree of control (I met with "we can't do that without changing the viewer" so many times). Second, tying back to the first reason, personally I think we are in a kind of a deadlock with the SL protocol. It was never designed for the sort of things the opensim community dreams about: rather, it is a very centralized, "the region server does everything" sort of protocol, that needs a fertile testing ground in the shape of a viewer that you can bend to your own will.

When exactly can you get the source code? I don't really have the answer to this question, as it is not nearly my decision. For now, just keep an eye on this viewer because we are going to deliver some features that I hope will show everyone that by changing the viewer, you can accomplish extraordinary things.

Features

The main distinction is of course that this viewer runs directly inside the browser. Now, does that mean, that it is forever constrained within the limits of a solitary HTML page? Not really.Internally we sometimes use a standalone version for testing, so technically there is nothing stopping someone to "liberate" (as I'm sure many would call it so:) ) the viewer from the evil browser. Our first target was IE only (boohoo, yes) and at the very least people can start using it they way they are already used to after years of experience with ye' olde series of tubes and, equally importantly, provide us with feedback on the important questions (no, not about our plugin system or our 3D engine). And the next stop is Firefox and Safari support.

The viewer itself is operated with the mouse only. This makes it simple and straightforward; on the other hand, there are not so many actions to do: you can walk around, run about, sit down to objects or touch objects. More complicated actions like chat or teleporting are of course in a simplified menu system. Our first priority is to make OpenSim - and virtual worlds in general - accessible to a wide audience. This will hopefully help the community indirectly as well as more and more people get used to using virtual worlds, more businesses will join the ride. Of course we could have gone to the other extreme and create a highly customizable meta-GUI system, but for the uninitiated first-time virtual world user, that wouldn't really help.

The SL Viewer supports prims, which is a great thing if you have a limited bandwidth. Or if you push every single bit through your region server. But realistically, getting professional content into SecondLife can pose a challenge because of this. As realXtend has also realized this before, 3Di OpenViewer also enables content providers to use their usual 3D content created with most major 3D modelling software like 3DS Max or Maya with little or no change. This should further lower the entry barrier to many CPs that don't really have the time and resources to invest in creating prims for all of their popular contents. If they can import their content as-is, the cost goes down and at the same time customer satisfaction goes up. In turn, there is an increase in virtual world penetration, that is good for 3Di as well as the community. (Hopefully the trend to make virtual worlds popular and more accessible can be seen here)

The last new feature for today is the JavaScript API: we are providing ways for a fair amount of intercommunication with a CP's HTML content and OpenViewer.You want to create an e-commerce solution in OpenSim supporting currency? Why go to the trouble of recreating your already existing EC in the virtual world, when you can quite easily communicate between the 3D world and the 2D world. The demo site is actually showing a mockup for an EC site that doesn't require lengthy configurations and recompilation of OpenSim. No, it's not an end-all-be-all, but it's a simple solution that one can use to quickly start up a new business in the virtual worlds.

In closing

There might be two things apparent from this post: first, I tried to convey why I think a new viewer, and specifically an in-browser viewer is good for the community. Second, I never talked about technical details. This latter, I am planning to ammend in a later post, but first I just wanted to make it clear that 3Di OpenViewer is not a product in isolation, and although it is only a first step towards a bright future for OpenSim and virtual worlds in general, I believe that it will be a significant one. So please keep an eye on the product, and do post any feedback you might have in the comments.

Voice chat in OpenSim

dezso.zoltan@gmail.com (Zoltan Dezso) — Thu, 28 Aug 2008 22:07:49 -0700

The content of this post is outdated. To learn more about the new FreeSWITCH connector in OpenSim, go to
FreeSWITCH module in OpenSim

Overview

If you want to set up voice chat for your grid, here is a rundown on what you will need and how it works.

OpenSim currently uses the Asterisk PBX server through a dynamic, MySQL-based configuration that is updated by a "frontend" python script. This frontend is invoked by an SLVoice replacement when a user REGISTERs to the SIP channel configured for the region.

For this, SLVoice requires two pieces of information:

The address of the "frontend"
The address of asterisk to register with

The region server must provide the asterisk address to the viewer, which in turns will pass it down to SLVoice. Similarly, the address of the frontend is set in OpenSim.ini and registration happens via the AsteriskVoice module in Region.Environment.Modules.Avatar.Voice

However for private chats - by limitation of the SLViewer - SLVoice always receives a hard-coded SIP URI that then needs to be changed to the real address. For flexibility this URI is queried at runtime from an HTTP server (the address of which needs to be set in slvoice.ini).

After SLVoice has registered through the frontend, it can contact the Asterisk service and start the real voice chat.

Installation

Zaptel

http://downloads.digium.com/pub/telephony/zaptel/

Zaptel is a kernel module that allows asterisk to use various telephony hardware, but for our purposes it is only needed for the MeetMe conference application that depends on a hardware timer. Because of this, the ztdummy kernel module is what we will need.

Zaptel requires that you have

bison, bison-devel newt, newt-devel ncurses, ncurses-devel

Installation is straightforward: ./configure; make; make install

(Note: You need to be root to install zaptel)

After installation, make sure you run modprobe ztdummy; modprobe zaptel

Asterisk

http://www.asterisk.org/

Before you start, please make sure the following are installed:

mysql, mysql-connector-odbc unixODBC-devel libtool-ltdl-devel kernel-devel

The latest versions of Asterisk don't include the sources for the codec iLBC anymore. If you want those supported, please run contrib/scripts/get_ilbc_source.sh

It is advised that you run make menuselect and configure the modules before you install asterisk. You will need res_odbc to interface properly with MySQL. Also make sure that ztdummy and app_meetme are marked for install. Then make; make install.

OpenSim

Get OpenSim and apply [Patch 1689] (http://opensimulator.org/mantis/view.php? id=1689) to get private chat working.

Asterisk Frontend

The current frontend requires Python2.5 with mysql support (http://sourceforge.net/projects/mysql-python if not included in the distribution) The frontend can be found in share/python/asterisk/

Configuration

Asterisk

In MySQL, create a database and add an 'asterisk' user by

    CREATE DATABASE asterisk;
    GRANT ALL ON asterisk.* TO asterisk@<host> IDENTIFIED BY '<password>';

The frontend will create the schema on first start.

Set up ODBC:

/etc/odbcinst.ini

[MySQL]
  Description = MySQL For Asterisk
  Driver = /usr/lib/libmyodbc3.so
  Setup = /usr/lib/libodbcmyS.so
  FileUsage = 1

/etc/odbc.ini

  [asterisk]
    Driver = MySQL
    Server =<server address>
    Database = asterisk
    Port = 3306
    User = asterisk
    Password = *********
    Socket =
    Option = 3
    Stmt =

/etc/asterisk/extconfig.conf

    sipusers => odbc,asterisk,ast_sipfriends
    sippeers => odbc,asterisk,ast_sipfriends
    extensions => odbc,asterisk,extensions_table

/etc/asterisk/extensions.conf

  [avatare]
    switch => Realtime/@

/etc/asterisk/modules.conf

  [modules]
    autoload => yes
    noload => pbx_gtkconsole.so
    load => res_musiconhold.so
    noload => chan_alsa.so
    preload => res_odbc.so
    preload => res_config_odbc.so

/etc/asterisk/res_odbc.conf

  [asterisk]
    enabled => yes
    dsn => asterisk
    username => asterisk
    password => *********
    pre_connect => yes

Frontend

In asterisk-opensim.cfg, set

  [xmlrpc]
    baseurl = http://<frontendhost>:<frontendport>

  [mysql]
    server   = <mysqlhost>
    database = asterisk
    user     = asterisk
    password = ************
    debug    = true

  [mysql-templates]
    tables = create-table.sql
    user   = create-user.sql
    region = create-region.sql

OpenSim

In OpenSim.ini, add the following section:

  [AsteriskVoice]
    enabled = true
    sip_domain = <asterisk address>
    conf_domain = <asterisk address>
    asterisk_frontend = http://<frontendhost>:<frontendport>/
    asterisk_password = **********
    asterisk_timeout = 3000
    asterisk_salt = paluempalum

SLVoice

Create an HTML page that has a single line stating the SIP URI to be used for private chats (same as asterisk address) Set this page's address in slvoice.ini and connect to the region. Voice chat should be working now.

What's next

There is an ongoing effort to refactor the frontend code. The resulting architecture will be much cleaner with fewer connections.
The new VoiceServer in progress already includes support for SIP proxies like OpenSER, the SLVoice replacement will be updated shortly to add proxy support as well.

Zaki.asia

Startup Weekend Notes

Day 1 - Pitches

Half Baked

It is not the Idea but the Team that makes a successful startup.

1 - The IDEA is the most important part of a startup. A great TEAM is the best way to it.

The pitch

2 - More than one pitch causes pitch fatigue. Make sure you have undivided attention.

3 - Everyone sweats it. You just have to push yourself.

4 - Tell a story. Sound confident. Involve the audience.

5 - A good pitch is a poor indicator of future success.

Day 1 - Team Building

6 - Startup Weekend is NOT a realistic simulation of a startup. Just have fun.

7 - Get the best people

Day 2 - Discussions

8 - Communicate early and often.

9 - There is a fundamental difference between a project and a startup.

Day 2 - Development

10 - Having fun is really important. Developing is fun for developers.

11 - When you are having fun, you can pull off impossible-looking feats.

Day 3 - Crunch

12 - Everyone has doubts just before the end. Just push through.

Day 3 - Final Presentation

13 - Make sure you have a compelling story. Product is secondary.

Day 3 - General

14 - Talk to everyone.

Rails 3 javascript_include_tag

Yaml !!merge

Testing Rails 3.0

Getting rails 3

Testing

activemodel

activeresource

actionmailer

activesupport

actionpack

activerecord

Referrer spam in statpress

C for Ruby

Introduction to C

Understanding Array#reverse

Extending Fixnum#succ

Writing native extensions for Ruby

F# Discriminated Unions

F# Currying

F# Swap

OSGrid in Rei

Textures

Avatars

OpenSim support

The current state

LBSA Plaza

Wright Plaza

Wii Controller Plugin for 3Di Viewer Rei

Wii Controller Plugin

How to connect the controller?

How to use the Wiimote?

About the demo

Plugins for Rei

Plugins

HEALTH BAR

DAMAGE

HEAL

DEAD

What do plugins do?

How to write plugins?

IManagerPlugin

Communication with Javascript

Plugin to Javascript: Dispatch

Javascript to Plugin: Callback

Plugin to Plugin: Message

Irrlicht.NETCP and 3Di Rei on Linux

Irrlicht.NETCP

Getting Irrlicht.NETCP

Rei and Irrlicht.NETCP

One more thing...

Intelligent evolution

Slowly getting there...

OSGRID IN 3DI OPENVIEWER

OSGRID IN HIPPO VIEWER