VSeeFace & OSC Motion Data

I’m completely new to all things OSC but I’ve been using VSeeFace as a poor man’s motion capture option. VSeeFace can both recieve and send OSC data. I’m able to caputre an OSC stream from VSeeFace onto the TWO Timeline, but when I press play to play back, TWO crashes. Every time. I’d love to explore the potential of TWO and VSeeFace working together, but I’m not sure if it’s even possible. Am I missing something? Any idea why it crashes? The clips definitely seem to be recording data so I’m hopeful there’s a fix. Thanks!

That’s not good at all, but the good news is a crash that consistently reproducible should be easy to fix, I will look right into it!
I’ve never used VSeeFace, didn’t know about it but I should be able to get the hang of it quickly enough.

It’d be helpful if you could tell me what OS you are on, what version of TWO you are running, and what would be most helpful of all is if you could save and send me a .two file with the recorded data - just record and save before pressing play and it should be fine :slight_smile:

Uploading it here should work, otherwise just email it to onar3d at gmail dot com.

Thank you for reporting this!

Thanks so much for the fast response! Please see my specs below. Will have to email the files as new users are not allowed to upload files :slight_smile:

TWO version: 0.9.14

Windows version:
Edition Windows 10 Home
Version 20H2
Installed on ‎9/‎23/‎2021
OS build 19042.1237
Serial number MJ0EZ8HL
Experience Windows Feature Experience Pack 120.2212.3530.0

Device name DESKTOP-MO4PSO8
Processor Intel(R) Core™ i7-10700K CPU @ 3.80GHz 3.79 GHz
Installed RAM 16.0 GB (15.8 GB usable)
System type 64-bit operating system, x64-based processor
Pen and touch No pen or touch input is available for this display

1 Like

Hi,thank you for the files!

I’m not sure about how VSeeFace will respond to having the entirety of the OSC stream it produces transmitted back to it - AFAIK no other software than TWO allows that (but I’d love to stand corrected), and I’m very nearly certain that the developers of VSeeFace have never tested the use-case.

I’ve seen this problem with Resolume for example - if you record everything it produces, and then send the entire stream back, it doesn’t react well, with jumpy/jittery movement, because it just wasn’t made/tested with that in mind!

There are workarounds, for example filtering out / removing certain messages (in the case of Resolume the high-density “position” ones), but with VSeeFace I’m not sure.

I’ve given it a spin with your files and indeed experience instability in TWO, it freezes, probably because of the high density of the data. I will definitely work on fixing that and am grateful for the test files!

Is there perhaps some VSeeFace project I should have too or do I just fire it up?

Last question, when you say it crashes, do you mean it freezes and becomes unresponsive, or a straight-up crash with a “segmentation fault” dialogue?

Best,
Ilias B.

Hello! Regarding the crashing, when I press play on the timeline, there is <second when the playhead starts to advance, then the entire window/UI simply disappears. The software completely shuts down. I don’t get any dialog box or crash report of any kind.

I emailed a generic VSeeFace compatible VRM model directly to you. This is a model I use to set up my system (but not the model I use for actual motion capture). Most of VSeeFace users use Anime style models for VTubing and vitrual interactions, but I’m not using it for that. I have an animated robot mascot for some children’s programming which I’m trying to create customized animations for. Because of this, I realize that my use case for both TWO and VSeeFace are not typical, but I think it still falls within the limits of their capabilities.

There’s no VSeeFace project for you to use, but if you simply activate your webcam, you’ll get motion to the head and shoulders and that should be enough to send on to TWO.

I’m going to spend my weekend diving into TWO’s documentation to explore it’s capabilities. It’s amazing software! Thanks for your help!

1 Like

Hi!

Unfortunately it doesn’t crash on my Windows machine with your test files.

I will do a few more tests to see if I can make it fail though, and if not, I’ll look into if I can send you a special build which writes a log file when crashing.

Meanwhile I see that VSeeFace generates a lot of data, about 60000 messages for 19 seconds - after finding a fix for the crash I’d be curious to see if it doesn’t support features to reduce the message count, to make working with it easier.

One last test which would be useful for diagnosing:
If you have a moment, open the file C:/Program Files/TWO/Examples/AudioVisualBach/AudioVisualBach 0.9.10.two, and press play in the Timelines view - does it crash there too for you, or does it work fine?

Hi!

Since I found I cannot reproduce the crash on my machine, I made a special debug build of TWO which you can download from here:

https://drive.google.com/file/d/1lraLqMjKf0gT-fIDMhQPSkuvbY8LAR6o/view?usp=sharing

It will be much slower than the release so don’t replace your normal install with it - instead you should unpack and run it from a temp folder.

When there’s a crash TWO will create two files in your user folder - mine is: C:\Users\Onar3D\AppData\Local - for your it will be different since your account will not be named Onar3D.

It will be two files, one named TWO_CrashLog_26 Sep 2021 11_34_05.dmp, the other TWO_CrashLog_26 Sep 2021 11_34_05.txt.

zip them up and email them to me!

(TWO will try to email them automatically, but depending on how your computer’s default email handler is set up that may or may not work).

Their content will help me diagnose what happens on your machine.

Thank you!

Ilias B.

Hello,

I was indeed able to playback the example files you suggested. Those seemed to open and play with no problems.

I forgot to mention that I actually installed TWO onto two computers. A Lenovo laptop and Lenovo desktop (the specs I sent you previously were for the desktop. Specs below are for the laptop I’m working on this weekend) Both of them crashed in identical ways, so it’s really interesting that your system didn’t crash.

Regarding the Crashlog install version you sent, unfortunately I got errors and was unable to install. Pics below of the error messages. Please advise.

thanks again,
Dave

Device name LAPTOP
Processor Intel(R) Core™ i9-10980HK CPU @ 2.40GHz 3.10 GHz
Installed RAM 32.0 GB (31.9 GB usable)
System type 64-bit operating system, x64-based processor
Pen and touch No pen or touch input is available for this display
Edition Windows 10 Home
Version 21H1
Installed on ‎9/‎26/‎2021
OS build 19043.1237
Serial number MP1TDP34
Experience Windows Feature Experience Pack 120.2212.3530.0

Screenshot 2021-09-26 191622
Screenshot 2021-09-26 191648
Screenshot 2021-09-26 191740
Screenshot 2021-09-26 191755

1 Like

Hello. I was wondering if you had seen this post. The log file version of the app won’t install on my system.

1 Like

Hi,

I’m very sorry, I completely missed these posts of yours, for some reason I never saw any notifications!

The errors you saw were because your computer does not have development tools installed, my mistake, I had forgotten separate C++ libraries were needed for that when Visual Studio isn’t installed.

I’ll look for a way in which those dll’s can be found and installed by end-users.

I’ll look for a non-dev windows machine I can test the TWO projects on as well!

Sorry again for this oversight of mine, I’ll make sure I get the notifications and check the forum in the future.

So, I set up a fresh Windows 10 Virtual Machine installation, and tried the example files you had sent me also on that one - unfortunately I did not get the crashing behavior you describe on that either.

Moreover I did some research on how to enable running debug builds of software on 3rd party computers, and found to my disappointment that Microsoft disallows the redistribution of the dll’s of the runtime in debug builds.

I’ll look into how to run a TWO build that can give a diagnostic stack trace without those, thank you for having tried!

Thanks for this. I’ve been doing more testing the last few weeks. I’ve also been testing things out on my intel iMac, OSX. That crashes at the same time and under the same conditions as my PCs so I’m consistently able to crash TWO, under the same conditions, across platforms.

Using Max 8, I’ve been able to filter out and limit the OSC data I’m sending to TWO and this has helped. TWO is still very unstable for me, but I’m able to record and playback limited OSC messages with verying degrees of success. But not to a degree that its usable. It’s just to a degree that tells me there may be hope down the road, once more of the bugs get worked out :slight_smile:

Please let me know how I can help continue testing, and if you have a preference for me to use Windows or Mac. There is so much potential here! Thanks again for working through this with me.

1 Like

Hi, thanks for getting back, and for persisting :slight_smile:

I’m on windows mainly, so that’s always easier to develop on - I use a virtual machine for the OS X building and testing.

What I DID see when running the test files you had sent a while ago, in my newly set-up virtual machine with limited access to CPU and ram, was that when TWO’s playback hit the big block of 60.000 messages, the GUI froze for a few seconds, while playback took over priority and CPU utilization hit 100%.

But after those seconds, the GUI would be released again, and the program would return to being responsive, I never got a crash as such.

Definitely do report other crashes you have had, I will make extra sure to keep an eye here, don’t know why I didn’t get the notification for your last messages but I’ve tested now and I do get them. If for any reason I seem unresponsive feel free to email me too!

My goal is as you say to get it stable now, while there’s always new features I could add, I think the current set would otherwise be a good “1.0”.

In my own use actually I don’t experience crashes, but that’s where other users such as yourself, who may use it differently, are so very valuable in finding errors I may not have.

Meanwhile, I think I’ve thought of a way to legally send you a debug build WITH those dll’s - I’ll send you an email with that when I have it working.

Regarding the point on using Max to filter out messages, if all you do is exclude specific address patterns completely, TWO has a feature for that already.

If you use Max for “rate limiting”, i.e. to reduce the numbed of message sent with a specific address pattern, TWO can’t do that - but it would be a nice feature to rate-limit inputs before recording. In that case let me know if that would be useful to you.

For the former, it’s as simple as clicking the “Mute Input” button on the input you want to exclude.

This can be done on an “Address Instance”, i.e. a particular node in the address editor. That mutes the messages only for that “Address” which uses the namespace, and is what I did in the test file I’ve uploaded below. You could also do it on the Namespace level, where there’s another mute option, but in this example I didn’t do that.

Edit - You can also mute at Node level, if you want messages to that node to be muted no matter what instance of it you are receiving at in TWO:

In the example file, I have recorded two clips to the timeline, one with filtering (on the AI level), one without. I also included a Max (7) patch which sends the messages. The Max patch depends on the OpenSoundControl external from CNMAT, probably there’s newer ones for that now but that’s what I had installed :slight_smile:

Filtering out messages before recording.zip (4.0 KB)

So, after more testing, I found and fixed two issues.

It was all thanks to using the files you sent - thanks again for those!

  • When recording / playing back very large numbers of messages, TWO would freeze. This was due to the message monitor window not being able to keep up. TWO now has an option to disable the logging in the monitor, which is otherwise triggered automatically when the message rate is very high, so as to avoid GUI freezing.

  • TWO would also freeze, if very large numbers of messages were being transmitted, playback was stopped, and then interaction with the GUI was attempted before the message backlog was emptied.

There’s a few more additions to v0.9.15, but those related directly to the issues you were having.

I seem to recall you’re using Windows - in which case the new release is available to download right away from here.

The Mac OS build is coming up.

Best,
Ilias B.

This is so great! I’m finally back online and downloading it right now. I’ll let you know how it goes. Thanks again!!!

1 Like