Learn how to write custom HTML overlay UI for VRED Core Stream interaction.
Video Captions: Hello and welcome to this tutorial for VRED Core. I'm your host Christopher and today I will show you how you can create a simple custom HTML interface for your VRED Core application.
VRED Core was developed to suit a variety of different needs. From providing a rendering pipeline, the data preparation, or acting as a dedicated real-time rendering server, it covers a whole range of use cases. Because VRED Core does not provide a user interface of its own, we are bound to communicate with it using the Python web interface.
VRED Core does come with a streaming app however, that gives you access to all functionality you would need for a product presentation or a design review. The streaming app enables you to access all variant sets and viewpoints in your scene, and also allows you to freely navigate with the camera. This is fine for a lot of applications, but VRED Core also offers tools that allow you to build a fully customized HTML user interface yourself. That means you can build an application that exactly serves your needs and can be brand with your corporate design.
In this tutorial, I will show you how you can build your own VRED app, including a live video stream, a simple Python terminal, and a list of your variant sets. To establish communication with VRED, we will use its Python endpoints.
We start by creating an HTML document that just contains a simple text paragraph. This document cannot be named index.html because VRED has already reserved that name. Therefore, we just call it myapp.html.
Next, we need to set up our HTML Web Interface preferences. This can be done by starting VRED Core with the command line parameter editPreferences. We switch to the Web Interface preferences tab and set the following settings. We enable File Access and set the directory to the directory where custom app is. At the bottom, we enable Allow Cross-Origin Requests. Please note the port number when you set it to something other than 8888 because we will need it to make Python requests to VRED.
These settings will initialize VRED's web interface and we can adjust the settings by running VRED Core and opening a browser with a port provided by localhost and the app, myapp.html. We should now see our example text and know that the web interface is working correctly. After that, we can switch back to our editor and continue to implement our app.
At first, we want to implement a live video stream with navigation. There are different ways to do this, but for simplicity, we can use the stream that is provided by the endpoint app's VRED stream. This endpoint provides a live video stream and a default navigation behavior we know from VRED Pro and Designer.
We can use this endpoint as a source for an iframe that we can configure to cover the whole page. To do this, we create a script, taken our document, what we'll add some javascript code, recreate a function, initializeStream, that searches for an element with ID stream container, and sets its source to our streaming endpoint. Here we are using the port number we set in the web preferences.
We call this function in the windowUnload function to make sure the stream is always initialized when the page is loaded. Of course, we also need an iframe element with a matching ID attribute that will show this video stream. With some CSS, we can define the size of the iframe to be 100% and prevent that any borders or scroll bars are visible.
As we load our app in the browser and see if our stream is working correctly. Nice! We now have a live video stream to VRED and can navigate the scene with the mouse.
Having a live video stream is a first step, but of course, we also want to implement some interaction in our app. For this tutorial, I want to integrate a simple Python terminal that allows us to send Python commands to VRED to manipulate our scene.
We start by adding a few new div containers that allow us to position the Python terminal over our video stream. Then, we add a text input and a button … the content of the input element to javascript function. I also want to show any return values that VRED is sending back, so I add that text area below the input field and button. We can style these new div containers so that the Python interface is positioned in the lower right corner of the browser window.
To send Python commands, we can create a new function in our javascript area, called sendAndReceivePython. And this function uses an XML HTTP request object to send the get request to the Python endpoint, pythoneval2. This endpoint accepts Python commands and also returns the result of the Python operation as a string.
At this point, I also want to sort a port and a host in separate variables that they are easy to change in the future. Be sure to encode your Python commands as an URI so that VRED can understand it.
We also use success and error callback functions to be more flexible. The successCallback will be called with response of our get request and the errorCallback will be called in case of an error.
Next, we implement the function executeTerminalCode that actually sends the Python command to VRED, and attaches the result message to our text area element. We just have to call the function when a submit button is clicked and pass the content of the input field as a parameter. When we wired everything correctly, we should be able to send simple print commands to VRED or request all variant sets from the scene.
The list of variant sets is also shown in our Python terminal window because VRED returns a list of them with a return message. We saw that we can get a list of all available variant sets in our app, so why don't we show them in the interface and make them selectable?
Let's create a new javascript function, called updateVariantSets. In this function, we call our send and receive Python function with the Python command, getVariantSets, and provide a callback method for the response.
For now, we just want to print the variant sets to the console. We call this function in our windowUnload function, reload the web page, and can open the console of the web page by pressing F12 in our browser. When you reload or open the page, you now see the list of variant sets in the currently loaded scene. When there are no variant sets, you will see an empty error here.
Here, I'm testing with the Genesis example from VRED that includes a variety of variant sets. Our app already receives all variant sets as a list, but we also have to add them to our HTML. In the next step, we add another div container with the ID variant set container to our HTML. This container will hold a list of all the variant sets that we receive from VRED.
Back in our updateVariantSets function, we first have to pass our variant set array as a real javascript array with the eval function, then we get a reference to our container with the getElementID function. We can iterate all the variant set array and create an HTML element node, and then append this node to our container.
We also want to be able to click on the variant set name and to select it in VRED. To do this, we can add a onClick function to the variant set node. Whenever a user clicks on a variant set node, we call the sendAndReceive Python function with the parameter, selectVariantSet and the name of the variant set. Then, we add some CSS to place our new variant set list on the page. This time on the left side.When we reload the page, we see a list of variant sets on the left, and when we click on one, the variant set is also selected in VRED.
In this tutorial, we built a fully customized streaming app from scratch with the help of a little HTML, javascript, and CSS. VRED Core offers multiple endpoints we can use to add a live video stream with navigation and change our request data in VRED with Python commands. There's so much more possibilities but covering all would get out of scope here.
In our next tutorial, we will extend our HTML app and implement some more features, using the new web API. That's it for today. Thanks for joining me and see you next time.
Here are the accompanying example Python scripts for the Tutorial 5: How to Write Custom HTML Overlay UI for VRED Core Stream Interaction video.
To download these files zipped, click here.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<title>My Custom App</title>
</head>
<body>
<p>This is my custom VRED App</p>
</body>
</html><!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<title>My Custom App</title>
<style>
html, body {
padding: 0;
margin: 0;
width: 100vw;
height: 100vh;
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif
}
#page {
width: 100%;
height: 100%;
display: flex;
justify-content: center;
align-items: center;
}
#stream-container {
width: 100%;
height: 100%;
overflow: hidden;
}
</style>
<script>
function initializeStream() {
document.getElementById("stream-container").src = "http://localhost:8888/apps/VREDStream/index.html";
}
window.onload = function() {
initializeStream();
updateVariantSes();
}
</script>
</head>
<body>
<div id="page">
<iframe id="stream-container" frameborder="0" overflow="hidden" scroll="no"></iframe>
</div>
</body>
</html><!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<title>My Custom App</title>
<style>
html, body {
padding: 0;
margin: 0;
width: 100vw;
height: 100vh;
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif
}
#page {
width: 100%;
height: 100%;
display: flex;
justify-content: center;
align-items: center;
}
#stream-container {
width: 100%;
height: 100%;
overflow: hidden;
}
.python-interface {
padding: 1em 2em;
position: absolute;
right: 1em;
bottom: 1em;
background-color: #2d2d2d;
display: flex;
flex-direction: column;
}
.python-send-wrapper {
margin-bottom: 0.5em;
}
.python-send-wrapper > input {
width: 25em;
}
#python-response {
height: 10em;
resize: none;
}
</style>
<script>
var host = "localhost";
var port = "8888";
function initializeStream() {
document.getElementById("stream-container").src = 'http://' + host + ':' + port + '/apps/VREDStream/index.html';
}
function executeTerminalCode(python) {
console.log(python);
sendAndReceivePython(python,
(response) => {
var text = document.getElementById('python-response').value;
text += '\n' + response;
document.getElementById('python-response').value = text;
},
(error) => console.error("Error sending pyhton command.", python)
);
}
function sendAndReceivePython(command, successCallback, errorCallback) {
var http = new XMLHttpRequest();
var url = 'http://' + host + ':' + port + '/pythoneval2?value=' + encodeURI(command);
http.open('GET', url, true);
http.onreadystatechange = function() {
if (this.readyState == 4 && this.status == 200) {
if(this.responseText && successCallback) {
successCallback(this.responseText)
}
}
};
http.onerror = function() {
if(errorCallback) {
errorCallback();
}
}
http.send();
}
window.onload = function() {
initializeStream();
}
</script>
</head>
<body>
<div id="page">
Variant Set Container
<div id="variant-set-container"></div>
Video Stream Container
<iframe id="stream-container" frameborder="0" overflow="hidden" scroll="no"></iframe>
Python Terminal Container
<div class="python-interface">
<div class="python-send-wrapper">
<input id="python-value" type="text" placeholder="Enter your python code..."></input>
<button onclick="executeTerminalCode(document.getElementById('python-value').value)">Send Python</button>
</div>
<textarea id="python-response"></textarea>
</div>
</div>
</body>
</html><!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<title>My Custom App</title>
<style>
html, body {
padding: 0;
margin: 0;
width: 100vw;
height: 100vh;
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif
}
#page {
width: 100%;
height: 100%;
display: flex;
justify-content: center;
align-items: center;
}
#stream-container {
width: 100%;
height: 100%;
overflow: hidden;
}
.python-interface {
padding: 1em 2em;
position: absolute;
right: 1em;
bottom: 1em;
background-color: #2d2d2d;
display: flex;
flex-direction: column;
}
.python-send-wrapper {
margin-bottom: 0.5em;
}
.python-send-wrapper > input {
width: 25em;
}
textarea#python-response {
height: 10em;
resize: none;
}
</style>
<script>
var host = "localhost";
var port = "8888";
function initializeStream() {
document.getElementById("stream-container").src = 'http://' + host + ':' + port + '/apps/VREDStream/index.html';
}
function updateVariantSets() {
sendAndReceivePython("getVariantSets()",
(response) => {
console.log(response);
}
);
}
function executeTerminalCode(python) {
console.log(python);
sendAndReceivePython(python,
(response) => {
var text = document.getElementById('python-response').value;
text += '\n' + response;
document.getElementById('python-response').value = text;
},
(error) => console.error("Error sending pyhton command.", python)
);
}
function sendAndReceivePython(command, successCallback, errorCallback) {
var http = new XMLHttpRequest();
var url = 'http://' + host + ':' + port + '/pythoneval2?value=' + encodeURI(command);
http.open('GET', url, true);
http.onreadystatechange = function() {
if (this.readyState == 4 && this.status == 200) {
if(this.responseText && successCallback) {
successCallback(this.responseText)
}
}
};
http.onerror = function() {
if(errorCallback) {
errorCallback();
}
}
http.send();
}
window.onload = function() {
initializeStream();
updateVariantSets();
}
</script>
</head>
<body>
<div id="page">
Variant Set Container
<div id="variant-set-container"></div>
Video Stream Container
<iframe id="stream-container" frameborder="0" overflow="hidden" scroll="no"></iframe>
Python Terminal Container
<div class="python-interface">
<div class="python-send-wrapper">
<input id="python-value" type="text" placeholder="Enter your python code..."></input>
<button onclick="executeTerminalCode(document.getElementById('python-value').value)">Send Python</button>
</div>
<textarea id="python-response"></textarea>
</div>
</div>
</body>
</html><!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<title>My Custom App</title>
<style>
html, body {
padding: 0;
margin: 0;
width: 100vw;
height: 100vh;
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif
}
#page {
width: 100%;
height: 100%;
display: flex;
justify-content: center;
align-items: center;
}
#variant-set-container {
padding: 1em 2em;
position: absolute;
left: 1em;
bottom: 1em;
background-color: #2d2d2d;
display: flex;
flex-direction: column;
height: 20em;
overflow-y: scroll;
color: white;
}
#variant-set-container > div {
padding: 0.1em 0.2em;
background-color: #4b4b4b;
margin-bottom: 0.1em;
cursor: pointer;
}
#stream-container {
width: 100%;
height: 100%;
overflow: hidden;
}
.python-interface {
padding: 1em 2em;
position: absolute;
right: 1em;
bottom: 1em;
background-color: #2d2d2d;
display: flex;
flex-direction: column;
}
.python-send-wrapper {
margin-bottom: 0.5em;
}
.python-send-wrapper > input {
width: 25em;
}
textarea#python-response {
height: 10em;
resize: none;
}
</style>
<script>
var host = "localhost";
var port = "8888";
function initializeStream() {
document.getElementById("stream-container").src = 'http://' + host + ':' + port + '/apps/VREDStream/index.html';
}
function updateVariantSets() {
sendAndReceivePython("getVariantSets()",
(response) => {
var variantSets = eval(response);
if(variantSets) {
// get a reference to our variant set container
var variantSetContainer = document.getElementById('variant-set-container');
// delete variant set list
variantSetContainer.textContent = '';
// add element for each variant set
variantSets.forEach(variantSet => {
var variantSetNode = document.createElement('div');
variantSetNode.innerHTML = variantSet;
variantSetNode.onclick = function() {
sendAndReceivePython("selectVariantSet('"+variantSet+"')");
}
variantSetContainer.appendChild(variantSetNode);
});
}
}
);
}
function executeTerminalCode(python) {
console.log(python);
sendAndReceivePython(python,
(response) => {
var text = document.getElementById('python-response').value;
text += '\n' + response;
document.getElementById('python-response').value = text;
},
(error) => console.error("Error sending pyhton command.", python)
);
}
function sendAndReceivePython(command, successCallback, errorCallback) {
var http = new XMLHttpRequest();
var url = 'http://' + host + ':' + port + '/pythoneval2?value=' + encodeURI(command);
http.open('GET', url, true);
http.onreadystatechange = function() {
if (this.readyState == 4 && this.status == 200) {
if(this.responseText && successCallback) {
successCallback(this.responseText)
}
}
};
http.onerror = function() {
if(errorCallback) {
errorCallback();
}
}
http.send();
}
window.onload = function() {
initializeStream();
updateVariantSets();
}
</script>
</head>
<body>
<div id="page">
Variant Set Container
<div id="variant-set-container"></div>
Video Stream Container
<iframe id="stream-container" frameborder="0" overflow="hidden" scroll="no"></iframe>
Python Terminal Container
<div class="python-interface">
<div class="python-send-wrapper">
<input id="python-value" type="text" placeholder="Enter your python code..."></input>
<button onclick="executeTerminalCode(document.getElementById('python-value').value)">Send Python</button>
</div>
<textarea id="python-response"></textarea>
</div>
</div>
</body>
</html>