Netscapes: Insight – IVT Testing

Today we did our final build and first live test in the Immersive Vision Theatre (IVT). We started by fitting the Raspberry Pi and touchscreen inside the plinth, then transporting the equipment to the dome ready for our presentation.

IMG_20180122_160933

Fitting Pi3 + Touchscreen

Chris added wooden beams to support the weight of the Pi, as it will be under a lot of pressure when the touchscreen is in use. This should prevent the touchscreen moving away from the pinth.

IMG_20180122_150137.jpg

Setting up in the IVT – Modifying Code

Whilst in the IVT, Gintare updated her code to work better within the shape of the screen. She moved some of the key elements of the visuals so they were more centered within the dome, bringing them to the viewer’s attention.

 

vizlaptop.png

Setting up the visualization

We transported the physical part of our project to the IVT and decided where to set it up. We then tested the project within the space to understand how it will look and feel to the viewers and how the colours will display in the dome.

head interface.png

Glass head with touchscreen interface

We took this as an opportunity to double-check our database connections were working. During this time we ran into issues with page refreshing (which I quickly resolved) and with internet connection, which we resolved by using a mobile access point.

headdemo.png

Glass head interface in front of the projection.

We even invited Luke to test out our user interface, and have a go at inputting his own data into the visualization!

head interaction.png

Luke testing out the user interface!

dometest

Head test with visualization within the dome.

Netscapes: Building Bluetooth Connections – Part 2

Today we had access to the physical side of the project, so I tested my Bluetooth code (see my previous post) with the Arduino side. Luckily, after pairing with the HC-05 Bluetooth component, the code worked first time without need for debugging!

IMG_20180122_160927

The Arduino side, with HC-05 Bluetooth component & Neopixel ring

Chris and I modified the Arduino code to output different lighting effects based on the character sent across Bluetooth. We decided on the default being Red, with a breathing effect (which I created for a previous project) and a rainbow spin effect.

LEDbluetooth

Bluetooth message sent on tapping “Generate”

How it works

  • When the local server is started, it searches through paired devices to find the HC-05 module.
  • When it is found, it opens a connection and sends it the instruction to turn on.
  • When the generate button is pressed, a new message is sent across the connection instructing it to run the rainbow effect.

Critical analysis/Reflection

To begin with, we were going to use a separate mobile app to input user data across Bluetooth to the Arduino. Switching instead to using the same input as the user data adds a level of interactivity than we would have previously had from a separate phone app. It allows a user to instantly see the effect their inputs have had even before the visualization updates.

This also ties the piece together better, making it an all-in-one system rather than being split up.

Future Improvements

If we had more time, I would modify the code to react differently depending on some of the user inputted data, such as changing colours or effects based on values.

 

 

Netscapes: Building Bluetooth connections

To bring together the visualisation and physical prototype, I started working on a Bluetooth connection to the MongoDB connection code I previously built.

IMG_20180113_141644

Physical prototype with HC-05 Bluetooth module

Since we already have the HC-05 Bluetooth module in place and working with the Bluetooth terminal input on mobile, I simply had to look up how to create an output system in our .js code to match the inputs we previously designed for the Arduino.

BSP design.jpg

Initial flow diagram of program

I looked into how this could be done and began researching into using Bluetooth-Serial-Port module for Node.js.

After getting to grips with how the library works, I experimented with creating a basic framework for opening a Bluetooth connection and sending a basic input.  This code will check for a connection with the correct name, find the matching address, open a connection, and if it is successful, and the character ‘a’. When hooked up to the glass head model, this should activate the LED ring, making it light up.

bluetooth serial build code

My experimentation with BSP within the previously made MongoDB connection code

 


Issues

  • Certain information missing from Bluetooth-Serial-Port NPM documentation – I had to work around this by searching for other uses of BSP to fill in the gaps
  • Method to call previously paired Bluetooth devices doesn’t work on linux systems, so a workaround has to be made (looping through available connections and matching a name)

Next Steps

  • Update Arduino-side code: Modify existing code to include more interesting light effects, such as those I previously created for my ‘Everyware’ project. These would not be direct copies, but modifications of this pre-existing code, for a unique lighting effect.
  • Thoroughly test this code to ensure a secure connection is made and maintained for the duration of the installation.

Code Referencing/Libraries Used

Below is a list of the code documentations I used as reference when building my code. Whilst code was not directly copied, it was heavily referenced from the documentation:

bluetooth-serial-port: https://www.npmjs.com/package/bluetooth-serial-port
JS express – https://expressjs.com/en/guide/routing.html
JS json body parser – https://www.npmjs.com/package/body-parser-json
JS path – https://nodejs.org/api/path.html
JS Mongo Client – https://mongodb.github.io/node-mongodb-native/api-generated/mongoclient.html

Everyware: The Icebreaker

Presentation Link: https://prezi.com/view/cvW3ewB4za09OKP7Hktq/

The Icebreaker T-shirt is a unique tool for connecting people. It is designed for use in conventions, open days, and other large social events. Plymouth University alone has approximately 23,000 students, with many of these students attending open days and freshers fairs. (University of Plymouth, 2018) With the phrase “It’s not what you know but who you know” being more true than ever; it is vital to grow your social circle and create opportunities for both now and the future.

26972401_10160029255395790_1364697445_o

Icebreaker mobile app

The Icebreaker mobile app accompanies the T-shirt, allowing users to input their login details as well as add in their hobbies and interests to match upon. The app is simple, allowing you to input your details and go, so your attention is always in the moment, rather than at your phone.

Watch our promotional video: https://www.youtube.com/watch?v=xVB_mYe-LQw

The interactive T-shirt makes the immaterial concept of shared interests and brings them to life in a visual way. When you meet someone you have similar interests to, your shirt will light up, telling you exactly who you have matched with and what interests you match on. The accompanying app tells you exactly how close the person you matched with is, so you never miss an opportunity to meet someone.

T-shirt designs

Icebreaker Tshirt designs

The Icebreaker T-shirt allows wearers to expand their social circle and meet new people – Perfect for university freshers fairs and open days.

Security features

Don’t want to match with anyone? Only want to match with people of a similar age? Don’t want to share your location? No problem. With the Icebreaker mobile app, you can choose from a range of privacy options.

Future updates include:

  • Personalisation: Customisation of colours and lighting effects based on preferences in the mobile app.
  • More Matches: A more extensive list of hobbies and interests to choose from.
  • Washable: Switching to washable conductive threads & removable LED panels, allowing the T-shirt to be reused

References

University of Plymouth. (2018). Facts and figures. [online] Available at: https://www.plymouth.ac.uk/your-university/about-us/facts-and-figures [Accessed 19 Jan. 2018].

 

Netscapes: Making & MLabs

Today we worked further on bringing the project together, drawing together all our current work and making improvements where necessary.

MLabs/Visualization connection

I worked on building a connection to the mLab database, pulling data and using them for parameters for a circle. The code checks the database for a new entry every 15 seconds.

readdb

Reading values from Database

For example, I set up mapping for sliders to RGB: The slider takes a value of 0 to 8 for the user, which is mapped to a number between 0 and 255 for 3 of the values (in this case the vars kind, trust and help). I also applied this to the radius and speed of movement.

Next, Gintaré and Chris will take this to build into their visualisation in its current state.

User Interface Modifications

We then looked at Gintaré’s slider inputs and how they would look in the physical build.

IMG_20180116_114315

First slider test in plinth (without the glass head or diffuser)

After reviewing both its looks and ease of interaction, we decided to make a few changes, such as making the text/scrollbar larger and removing the numbers from the sliders (As they do not display properly on Raspberry Pi).

Gintaré made modifications based on these observations and we quickly reviewed it. We also decided to colour code each section of sliders to each section of the CANOE model. This not only breaks it up but makes it more visually appealing in a way that makes sense.

IMG_20180116_120335

Touchscreen with enlarged scroll bar for ease of use.

We decided it would still be best to display the touchscreen with the stylus for ease of use as the sliders can still be difficult to use at this size.

IMG_20180116_123645

Touch screen with colour coded sections (per canoe model)

Since the touchscreen has no enabled right-click function, once the app is full-screen it is very difficult to get out of – meaning the viewers won’t be able to (intentionally or accidentally!) exit it.

We decided to bevel the edges that surround the screen as they make it difficult for users to easily reach the screen. This will also make it look more inviting to a user by bringing it into their view.

Connecting MongoDB/mLab to front-end

I started working on code to input values to the database using Gintaré’s previously made slider interface. This was built using express, npm and node.js. On recommendation from Chris B, Express was used in place of PHP.

When run, the code hosts the necessary files (such as Gintaré’s sliders) on a local server, which sends the data to the remote server when “Generate” is pressed.

 

Since Node.js means the code is ‘modular’, we decided to put the login details in a separate .js file (rather than censor the mongoDB login details when on GitHub)

IMG_20180116_171936

Installing Node.js & npm to Raspberry Pi

Once this was up and running (and confirmed to work on mLab), I moved the files and installed the necessary npm packages on my Raspberry Pi. I then tested the connection to mLab to ensure the data was working.

IMG_20180116_185022

Running the local server (Hosting the sliders form) on Raspberry Pi

We then put this server connection together with Gintaré’s updated user interface.

data canoe test

Data inserted into mLab via Raspberry Pi

mlabs multi canoe

Multiple documents in MongoDB database.

Now that we have data both coming into and out of the database, we are ready to move onto the next steps!

Next Steps

  • Finish Visualization
  • Put together final physical prototype (Seat raspi, sort out power supplies .etc)
  • Preview in IVT – test visualisations before presentation
  • (If time allows) Make a system for colour of head based on last data entry.

Netscapes: Building

Today we focused on finishing developing the physical side of our project.

ledhead

The LED ring powered glass head. Colours are being chosen by a mobile phone (off screen).

The second Neopixel ring arrived, so we soldered in some headers and built a circuit board for it. We installed the necessary drivers into the Arduino IDE and programmed it to output different colours.

We then merged this code with the Bluetooth code Chris made earlier.

We mounted the Arduino, breadboard and Bluetooth module to the interior on the plinth. We drilled holes into the head base to accommodate the wiring for the RGB LED ring.

IMG_20180113_161942

The Arduino & Circuitry mounted inside the plinth.

This LED ring is brighter than the other, even inside the diffuser, so is even better for our finalised build!

IMG_20180113_151014_1.jpg

The LED ring mounted in the plinth.

Our next steps are to:

  • I will develop an app to send characters across a Bluetooth connection, enabling us to remotely change the colour of the head without the need for the Bluetooth Terminal.
  • I need to build server connections for our visualization, input sliders and MySQL Database.
  • Gintaré and Chris need to complete the visualization for the IVT theatre.

 

Netscapes: Development Process

Creating a project is an organic process that contains many twists and turns. Below I will outline some of the changes we had to make during the development of our project.

Before Christmas break, Chris built a wooden plinth to mount the glass head on & house all the electronics. He also designed & 3D printed an inner diffuser for our lighting. This will be displayed in the IVT theatre, as the interactive front-end of our project.

IMG_20180109_164348.jpg

Glass head mounted on plinth (without diffuser). The gap at the front will house the Raspberry Pi & GPIO Touchscreen.

Modified Slider/LED control for Arduino

To further improve the LED lighting part of our piece, we decided to modify it by removing the serial connection and instead using a Bluetooth connection. Chris purchased a Bluetooth module and began to program it to take inputs from mobile.

Chris and I worked together to program the RGB LED code with Bluetooth. We tested the connection using Bluetooth terminal on our Android devices; sending simple “a” and “b” messages to turn an LED on and off remotely. We discovered that this will only work with one device at a time, so we will need to account for this when the model is on display.

We decided on making a mobile app to control the colour of the LEDs, which I will build in Processing over the next few days.

Resolving LED brightness issues 

We found that with the 3D printed inner diffuser in place, The RGB LEDs were not bright enough to light up the glass head.

IMG_20180109_151940

Original setup, with multiple RGB LEDs, Arduino & Bluetooth module.

img_20180121_200116.jpg

Neopixel 24 LED Ring

We tried an LED ring (that I have been using in another project) since it is considerably brighter than the individual LEDs. This worked much better; the colour was visible even in the brightly lit room, and the ring diameter was a perfect fit for the base of the diffuser!

IMG_20180112_132148

Glass head with diffuser and LED ring.

We purchased another LED ring and cut new holes in the mount to accommodate the wiring.

Switching and Setting up databases.

Due to issues connecting to our MongoDB database, we decided to switch from MongoDB to MySQL.

I set up a new database on my server with access to the necessary tables. I sent Gintaré a link to instructions on how to set it up, along with the necessary details, so she can get to work building the data visualization.

Next Steps
Our next steps are to:

  • Wire up the LED ring and program it to respond to Bluetooth messages (modifying earlier code)
  • Develop an android app
  • Connect the visualization and the slider inputs to my server/Database.

 

Everyware: MQTT matches & Lighting effects

Gerrit redesigned the T-shirt from the original polar bear design to a design which will help alert users to which interests they matched on, as seen below. This will match the size of the LED ring once it is fitted onto the T-shirt, allowing a quick visual reference to which interests or hobbies the wearers have matched on.

received_10159835159540181 (1).jpeg

Gerrit’s T-shirt design, made to accommodate the LED ring within the design

Since my part of the project includes handling the physical prototyping with NodeMCU, I had to change my code to work with this design.

Creating Lighting Effects – Neopixel 24

The new T-shirt design incorporates a thirds system to allow users to identify which interests they matched on.

IMG_20180116_212159.jpg

For single matches, only specific parts of the board will light up.

I changed my code to accommodate this thirds system – I split the LED ring up into thirds relating to which match was made. Since the Neopixel Library allows you to set individual LED colours based on their number, it is possible to count around the ring between specific numbers.

Although the LED ring will need to be ‘reset’ to off between animations, the left/right match code can otherwise stay the same.

ledring

Basic match with direction

This visual code works not only for the wearer but for other wearers too – It acts as a quick visual reference to allow you to instantly see who you matched with!

I also made my own functions for lighting effects such as breathe effects and directional matches. The breathe effect cycles through brightness as a percentage (in a similar way to setting the brightness in the code setup). Whilst this works, it does require perfect timing on delays, as if you miss the timing the LED ring will not go back to full brightness. Percentages should also be used as to not overwrite the initially set brightness of the LED ring (to avoid it turning down to ‘0’ brightness and staying there!)

Final Lighting Effects

I modified my code to show more interesting lighting effects and fix some issues that I previously came across.

One of the main issues I noticed was that when a match was made, if the ‘background’ was left as the default blue the matches were not as easy to notice (for example, if two matches were made, the last third would stay blue, so it could  be interpreted as 3 matches instead of 2). I combated this my programming the LEDs to switch off after matches and between directional animations. This also has the added bonus of being more eye-catching to both users and viewers!

effects.gif

Double match with direction effects

Matches count across one third of the ring (always in sequence, so the motion is a constant clockwise direction for a smoother appearance) If there are multiple matches, this will count across all of the thirds in sequence (as seen above). Once it has finished counting round, the ring lighting will “Breathe” for one cycle.

After this, the directional lighting will begin. This lights up half of the ring white and green. The effect means that the ring will appear to light up white with a green “arrow” moving towards the center in whichever direction the match is. To achieve this, I programmed the LEDs individually to delay the change in colour. This animation will run for 4 cycles.

Handling Multiple devices across MQTT

In order to keep it efficient, each board takes a unique ID and then an instruction, then a direction. for example “1” for board 1, followed by “1” for match one and “2” for match right. at the same time, board 2 would receive “2” followed by match “1” and “1” for match left.

tshirttable

MySQL database, where the interests are stored

When this is hooked up to Glenn’s MySQL and Node-Red code, these sets of instructions are sent to the relevant T-shirts when a match is made within a certain location. This location is worked out using the Google Maps API (also used to determine the Left/Right directional matches).

simmatch

2 LED rings matching on all 3 interests. Note the motioning towards each other. (Delay due to internet!)

 

Everyware: Icebreaker development – LEDs & MQTT

In this post I will briefly outline my part in the creation of our wearable technology: The Icebreaker T-Shirt.

My responsibilities included creating the physical prototypes: Using the NodeMCU and MQTT connections to build a wearable system.

Prototyping: Single RGB LED

I started off by prototyping MQTT connections with a single RGB LED. I built a basic circuit and programmed it so that it would connect to the MQTT broker, and light up when data is received.

This was a very basic prototype to test MQTT connections and reactions, to see what is possible across MQTT communications and how messages are both sent and received.

During this time, I ran into many internet connectivity issues with the NodeMCU board – Whilst it does work, it often needs resetting as the connection is prone to drop out after running for a while. This is a known limitation of the board and a workaround should  be found for the finished product.

IMG_20171125_192722

NodeMCU wired up to the RGB LED for testing messages sent across the MQTT i-dat broker.

RGBW LED Strips

After this, we experimented with using RGBW LED strips. I wired up a circuit (using transistors to prevent burning out my controller board) and experimented with some code to create patterns and colour combinations.

img_20171121_105614.jpg

Testing the RGB LED strip

In addition to issues with powering these strips from the NodeMCU board without external power sources, it was apparent these strips were not composed of individually addressable LEDs, so turned out to not be appropriate for our usage. It would also be difficult to fit these LED strips around a T-shirt design without cutting them in multiple places and soldering many wires to bridge the gaps, resulting in a messy and impractical finish.

img_20171121_122214.jpg

Wiring up the RGBW LED strip with multiple transistors – one for each colour & white.

 

24 RGB LED Ring

Next we moved onto using an Adafruit Neopixel 24 LED ring. First, I soldered some header pins into the ring for easy wiring. Later these could be removed and either directly soldered onto, or even potentially be connected to using conductive thread.

IMG_20171201_112748

24 LED ring with sample code.

I worked out how to individually address each LED to both change the colour and brightness, and then used the code from the first single RGB LED (shown above) to connect it to the MQTT broker.

After this, I setup basic reactions to each input to simulate what will happen once the database is properly connected; such as changing the colours when a match is detected.

Basic lighting effects/reactions i initially created include:

  • Ambient (No match/resting)
  • Match (Different colours for different matches) – Currently 3 max
  • Left – Left side lights up
  • Right – Right side lights up
img_20171210_210009.jpg

Wiring up my NodeMCU board to the LED ring. The circuit requires the use of capacitors to prevent damage to the LED ring and resistors on the data inputs. (Tape shown over LEDs is for diffusing)

The next steps are to create more visually appealing lighting effects & reactions (which I will work on over the next week) and to hook it up to the database via MQTT, which is being handled by my other team members.


 

Links

Adafruit Neopixel/Library: https://learn.adafruit.com/adafruit-neopixel-uberguide?view=all