Author: Duane Johnson

How we look and how we feel about ourselves is intimately connected. In Relm, we want you to feel at home, and firmly in your own shoes while building and being part of your community online.

To that end, we’ve put a lot of thought into character customization, and the technology that goes into it. This is part 1 in a series of posts explaining why and how we’re making customizable characters for you. And if you’re building similar technology, we hope you can benefit from our lessons learned.

The Low-Poly, Low-Bandwidth Constraint

One of the major constraints for character design in a web browser environment is the size of the character assets. We want the world to load quickly, and we want it to render at 60 fps for people with regular phones or computers (let’s say, 2015 or more recent).

We could design a whole bunch of characters to give variety and self-expression to people who take up an avatar in Relm; however, each character would be a new asset file and a new set of data to download for each participant. If there are 100 characters to choose from, that means everyone in the world needs to download 100 character assets so they can see everyone else’s avatars.

One Character, Many Colors

Instead, it would be ideal if we could create one character design that lets a participant choose their character’s colors: skin color, hair color, clothing colors, etc.

The challenge is that a 3D model has many surfaces (“faces” in 3D modeling lingo) and vertices, but it’s unclear to the game engine which surfaces or vertices belong to what “parts” of the 3D model. If we want the player to be able to change their skin color, for example, the game engine doesn’t know how do differentiate surfaces that are skin from those that are clothing. We need a way to “annotate” the model and say “these vertices belong to the feet” or, “this surface belong to the torso”.

Blender already has the tools we need to create the type of 3d model that would enable color customizations:

• colored vertices
• vertex groups, and
• face maps

However, exporting to glTF (the native format supported by three.js) doesn’t necessarily keep all of that information. The current built-in glTF exporter throws it away.

We needed to add some capabilities to the glTF exporter, as well as to the Relm engine (which uses three.js), so that participants can select colors for their avatar’s skin, hair, clothing and other aspects of self-expression.

Custom Blender Exporter

If you download & install the following addon to Blender (replacing the default glTF Import/Export addon), it will include Vertex Group & Face Map metadata from Blender in the glTF (or .glb) files you export:

If you’re new to addons, see this tutorial on how to install the addon in Blender.

The source code is also available if you’d like to see inside. Our Github repository is a fork of the official glTF-Blender-IO addon, and is available at https://github.com/canadaduane/glTF-Blender-IO.

As you can see in the Blender screenshot above, you can create named Vertex Groups or Face Maps. Vertex Groups can be used for coloration, but we found that they were less than ideal because a colored vertex will “blend” its color with adjacent vertices. In other words, you won’t get clean lines of separation between colors as you’d want between, say, a white shirt and a black belt (it would be white-grey-black).

We recommend using Face Maps for custom colorization because you’ll get both a slightly smaller file size, as well as clean, separate colors from face to face (see “viewer” below).

If you’re interested in the technical details:

• Each Vertex Group is added to the three.js geometry attributes as a _vg_[vertex_group_name] custom attribute (e.g. you can access the four Vertex Group shown in the screenshot as mesh.geometry.getAttribute('_vg_four')). It contains what is essentially a “mask” where 0 means a vertex is “not in the set” and 1 means a vertex is “in the set”. Thanks to scurest‘s work, the exporter is smart enough to use a sparse index if there are just a few vertices in the group.
• All Face Maps are added to the three.js geometry attribute as a single _facemaps attribute. Each vertex is given a Face Map group ID from 0 to n (where n is the number of Face Maps being exported). If a vertex does not belong to a face with an assigned group, it is given the value -1. There is an additional userData.facemaps property in the userData field of the mesh that contains the geometry. This property holds the names of the Face Maps, in an array. For example, ["top", "bottom"] would mean that Face Map group 0 is named top and Face Map group 1 is named bottom.

See this Github issue for further details.

Custom glTF Viewer

https://github.com/canadaduane/three-gltf-viewer

• 

• 

Don McCurdy’s glTF viewer is the gold standard in knowing, “does this glTF work in three.js?” However, in order to support these custom attributes, we needed to modify the viewer to see the _vg_[vertex_group_name] and _facemaps attributes and teach it how to use them to color a section of the model.

You can clone or download our custom glTF viewer and run it locally on your machine. In addition, you can try it out here if we still have it running (no guarantees on this fast-paced project, sorry!)

https://viewer.relm.us

Putting it All Together

With Vertex Groups or Face Maps as annotations for various parts of the character model, your character designer can annotate the “hair”, “face”, “shirt”, “boots” or whatever else you need to be able to color as separate parts. Then, the game engine can tell which parts of the model are semantically separate and take the vertices’ colors and lerp them towards a new target color (e.g. change the hair to “blonde”).

Finally, using these basic building blocks, we’ll be able to design a character creator in Relm. Next, we’ll explore how to use morph targets and animations to make the character have different body shapes and genders, all while being able to re-use the same animation data.

We can’t wait for you to be able to express your presence the way that you would like to, and be with others in the Relm community!

Ok, I just have to say I’m really proud of the work we’ve accomplished in 3.5 months. Relm 5.0 wasn’t even a line of code on Oct 28 of last year, and we’ve now made more than 250 additions and improvements to the codebase:

https://github.com/relm-us/relm/blob/main/CHANGELOG.md

Not to mention the many MANY worlds Rey has created, in addition to art, prototypes, mockups, ideas and discussions we’ve had. Thanks @Rey, @ChrisRelm, @Brendan, @Hira, @AlexRelm, @avawoolf and @dane!

Some of the things I’m very proud of having achieved in Relm 5.0:
– Editable 3D HTML labels
– Embedded youtube videos and 3D web pages
– Physics engine with colliders and joints
– The world (each relm) as a shared editable “document”
– Drag-drop world building library
– Flexible object primitives editor
– Very fast rendering (for a browser!)

Needless to say, I’m extremely excited about the next month or so. Milestones that I look forward to include:

1. Portals between relms (this time without reloading!)
2. Multiplayer (shared positional presence)
3. Video bubbles with Hira’s honeycomb algorithm
4. Multiplayer Physics

With the above 4 milestones in place, we’ll be ready for some really cool innovating in virtual locality–how to build community and spiritual-lite practices in a virtual space.

Should we have a “game mode” and a “builder mode”? What does private messaging look like? What does your character’s inventory look like? Is inventory separate from uploading a photo?

We’re discussing the new user interface and we have a lot of ideas!

Rey’s UI sketch is a “whole picture view” of things, trying to integrate many of the ideas we’ve had–a personal profile, a minimap, a thought bar, uploading photos, emojis, and then panels on the left including contact list, inventory, message inbox, builder library, and settings:

My (Duane’s) prototype focuses more on the “content creation” side, using a horizontal expanding panel, or accordion. It includes writing, expressing emotion as emojis, choosing existing photos or uploading new photos, and drawing:

If you’d like to join in the discussion, visit us on Discord. We love meeting other creative minds!

We’re happy to announce a new open source (MIT licensed) library called jitsi-svelte, available on npm and github.

I’ve been using lib-jitsi-meet for a few months now and have a little bit of an understanding of its API. A little while ago, we switched to using svelte as our HTML framework in Relm and I’ve absolutely loved it.

So, naturally, we needed a way to bring these two awesome pieces of technology together. Now, the jitsi-svelte library allows you to easily create your own custom Jitsi client, in the Svelte framework.

There are quite a few intricacies to getting video conferencing working well, and if you’re writing a Svelte3-based app, there is some additional complexity in getting all of Jitsi’s events to work together with reactive stores.

Jitsi-svelte simplifies all of this and provides the svelte stores you need to build a web-based video conferencing app. It also provides Svelte components for Audio & Video, among others.

Additionally, users expect to see themselves before entering a video conference, and ideally, need the ability to set up their cam and mic hardware as well. We were inspired by Whereby’s intro screen and created what we call the “Mirror” component that mimics their UX & design.

If you’re developing a svelte-based video conferencing app, check out jitsi-svelte and see if it fits your needs.

Our online spaces need to be more welcoming—they need to have more hospitality, more civilization, and more heaping helpings of kindness. You might be skeptical that there’s hope—after all, the social networks are most often a crap shoot: opening Twitter, for example, is more like “joining the fray” than feeling connected with warm friends.

Relm is a radical attempt to change what’s possible online. We understand the deep responsibility we have to our generation’s psychological health & well-being, and we are building a platform where human connection can grow, even through the thin wires of the internet that now connect our society.

A friend asked today how Relm is a kinder, gentler community. There are a few things we’re working on:

• Relm is audio on & video-enabled by default—so there’s a more human side to people. We think this is a necessary (but insufficient) requirement to enable our evolved sense of empathy towards each other as human beings.
• We’re simulating having a body, like in a video-game world, which enables more forms of communication. For example, sitting down with someone indicates more focused attention, or walking away can indicate boredom (among other things).
• [future work]: There will be a lobby that allows an anonymous person to enter a relm and get “vetted” before entering. Similar to the role a front desk plays at an office building or theme park.
• [future work]: We want to create a sense of ritual around entering a community, so it doesn’t feel bureaucratic—like your entry depends upon filling out a form correctly (e.g. “login”, “password”). Instead, we want it to be about making a commitment to people—someone (ideally, a friend) gives you a tour and explains the rules of conduct. (We use cryptographic keys generated in-browser to avoid the login form).
• [future work]: We’ve thought a lot about autonomy vs. control. In some situations, there are stricter expectations around control—imagine an auditorium with 100 people ready to listen to a presentation, for example. We’d like to simulate capabilities similar to civilization in the real world in these cases–a security officer, for example, that has the ability to escort people out if they’ve crossed a line (but we don’t want this to be “first line” tactics, because sometimes disturbing the peace/heckling is a legit form of expression/free speech).

These are just some ideas, but I hope it outlines some of the work that’s going in to making Relm a kind, gentle place where online communities can find health & happiness together. If you’re interested in joining us, come by our Discord channel and chat!

In some ways Relm is like Zoom (video chat!), but at the same time, a lot of people notice it also looks like… a video game world!? So, what’s up with that, and is it better than Zoom?

Video game worlds are fun, and multiplayer campaigns are so great for building team spirit and camaraderie. If that’s the case, we wonder, then why not use them in our everyday work? Why confine ourselves to bland video chat software?

Today, there are a few things that we think already make Relm better than Zoom:

• Have you experienced Zoom fatigue? We think it’s so unnatural to be placed with faces towards one another for an hour or more. Having the freedom to move yourself around, shift your attention, and meet in different settings is a big improvement over traditional video conferences–you won’t be as tired, because your brain is naturally novelty-seeking and you’ll find video chat in Relm refreshing.

• Has your team started using Zoom backgrounds for fun and self-expression? The fact that so many of us take advantage of this points to the need for self-expression. In Relm, you have many more ways to show your mood, tell a story, or create an environment that strikes up conversation. Relm is like a “shared team background” that can be the starting point for team or community culture.

• When you start up Zoom, you’re always in the same place–facing each other, with your office or synthetic background behind you. In real life, the environment we meet in is a huge social cue that normally constrains what can be talked about. When inviting someone to your home, this signals something far more intimate than meeting in the office. In Relm, you can build environments that allow many types of constraints and levels of intimacy for conversation.

But this is just the beginning! We hope to build a real global village, an open source platform where you can be you, where you can feel belonging and status in a healthy, human way.

Relm is attempting to combine a kind of social network (like Facebook) with video conferencing software (like Zoom or Whereby), with an MMO video game world (like World of Warcraft).

It’s an ambitious project, but one that we feel inspired to attempt today in this work-from-home, school-from-home, everything-from-home environment. We are building the online space that’s worthy of our spending time there. If this is inspiring to you, contact us!

Version 1.0

Human beings are fundamentally social creatures. But the world wide web was not designed to meet our social needs. When we look around at the networks we’ve created online, we see a travesty of real connection with each other—sometimes an emotional wasteland filled with failed efforts to see and to be seen, to be with and to belong.

Today, we have Facebook “friends”, Instagram influencers, LinkedIn connections, and Twitter mobs. In addition, we see more depression, anxiety, and loneliness in our society than ever before.

But if we can re-imagine the web the way it should be—not as an inter-linked store of hypertext documents, but as a place to work together and build community together—why not fashion for ourselves an online universe that is pro-social and social-first?

We evolved in a spatial world, and we thrive in 3 dimensions. Video games and MMO worlds have led the way in showing us how to build trust and culture online—and we should take their lessons seriously enough to integrate the experience they offer in fields as far away as remote teamwork and business meetings.

Our surroundings tell us about ourselves, and hold us in relationship to one another. As we work, create, and collaborate together, we need a virtual world in which to do it—not necessarily because it’s efficient, but because it’s the most human way we know to be online together.

We believe that the architecture of the web experience needs to be re-designed for online teams and communities. A healthy online universe for human beings prioritizes:

• Belonging over status updates
• Visual and auditory communication over textual communication
• Real-time interactions over asynchronous requests/responses
• Rootedness in community over fast network growth
• Hospitality over bureaucracy (e.g. login forms)
• Opportunity for human connection (& serendipity) over efficiency
• Socially meaningful surroundings over missing context or sterile environments
• Representing ourselves as avatars over having little to no representation of “me”
• Fun throughout!

The next version of the web experience should be a social universe—a place where we can see, be seen, and belong—just like our ancestors’ communities, but online.

But I’m about to wear several, for now!

I’m thrilled to announce that I (Duane) am starting full-time this week at Relm.US Inc. as CTO and acting CEO. When my brother, Chris, and I started this video game-like virtual world for teams last Christmas, we had no idea how important it would become in 2020 to help people feel connected online.

What is Relm? Imagine if Zoom and Animal Crossing had a baby. Let people customize their avatar and environment, then bring in business apps like whiteboards and kanban boards. All in a fun, human-connecting, open-source world.

As more and more teams, conferences, and classes have begun experimenting with flex work and hybrid online arrangements, it’s clear to us that although it’s wonderful to see collaborative tools online, tools are not enough. We need an environment to surround us and give us context as we work together–we need a place that grounds us, and gives us human connection throughout the day.

If you’re interested in learning more or if you’re seriously passionate about this kind of new development, let’s talk!

Most of the development work has been focused on a server update that will be coming soon, but there have also been some helpful updates and improvements that have landed on the browser side:

• NEW: Preliminary support for packed-glb and packed-gltf formats. See command-line gltfpack utility. This reduces some meshes and animations to 20% of original size.
• NEW: Selecting a 3D object now highlights its edges in black.
• NEW: Documentation (“Help”) now shows thought bar in command descriptions
• NEW: When selecting a single item, a toast message now shows you what you selected (type and ID)
• CHANGED: The upload limit has been changed to 4mb.
• CHANGED: On Mac OS, you can now Cmd+click to remove selection (ctrl-click is indistinguishable from right-click)
• FIXED: Restore landing coords to working state (x,z params in URL)
• FIXED: Uploading a 3D object used to only accept a single mesh; now all meshes in a glb/gltf file are accepted.
• FIXED: Documentation shows /share command for screen sharing
• FIXED: You can now drag objects that are “underneath” other objects, if the under-object is selected.
• FIXED: Players would sometimes get caught in “infinite reload loop” when triggering portal to other relm.
• FIXED: Players would sometimes “moonwalk” (from other players’ perspectives) rather than instantly teleport when using portal.

The Open Source Ecology group has a cool use-case for Relm: since they build actual life-size machines, they can collaborate in virtual space and coordinate builds. They requested the ability to highlight edges of 3D objects in order to better show off some of their CAD models.

An oft-requested feature is now here: screensharing!

If you click on someone’s screen, you’ll get a full screen version of their screen.

For now, the only way to initiate a screen share is via the command /share. Sending the /share command again will exit screen sharing mode and return to regular video.

Would seeing each others’ screens side-by-side be useful for a follow-along tutorial? Maybe as a way to show what you’re working on with the team? We’re excited to see how people use this new capability!