Here are my 3 scenarios which I feel Time Browser 1.0 should enable. I am not saying all of this needs to work, but the architecture should support it.
I would prefer that the system is built as a native application, for OS X and Windows if we must, enabling powerful local manipulation and DB synch.
Recording is not part of this description since recording should work to make these interactions possible and thus should be shaped for the scenarios agreed. A particular note is that it is clear that the recording would need as much automatic tagging as possible and also manual tagging, to provide as rich as possible interaction opportunities for currently foreseeable opportunities and to lay the foundation to future interactions.
Query the record for an utterance or a response
In this scenario I expect the user to be able to query the system via voice, if we can have this tech from Adam. At the very least we should be able to make an interface for this with pop-up options and text entry boxes. The results should be shown as a text document with deep interactions:
Show me everything Ed has said this month. This is the most basic query, show phrases based on a keyword. The time limitation is optional, it should be possible to also ask to see everything Ed has ever said into the system.
What did Lisa say about OHS this week? This is also a pretty basic query, show phrases based on a participant and a keyword.
What did Sam say in response to Stan about DKR? Results should be shown in the document as text, showing all text uttered by Sam after Ed was speaking and where one of them mentioned DKR.
Show me every time Bjørn agreed to something. In this case some basic semantic analysis needs to be done, in order to show sentences with ‘I agree’ and also ‘sure, why not’ and so on.
Select text and change the view
The visual interface is that of a text document, very much like Author (hey, it’s my aesthetic, let me push for it…). The user should be able to select text and:
Press Space Bar to play back the recorded audio for that selection of text. (how it’s done is up for discussion, I just currently like the space bar idea). This audio is then played back in stereo, where the users preference for who is where in stereo space is editable by re-ordering the list of speakers. This means we need a place to put information about the full document, so we will likely need a header section for the time and place the recording took place and who was in it.
Ctrl-Click (or something similar) and on a participants name and choose:
• Show me only text from ‘name’
• Do not show me text from ‘name’
• Show me who most speaks before ‘name’
• Show me who most speaks after ‘name’
• Show me who ‘name’ most agrees with
• Show me who ‘name’ most disagrees with
• & further queries along the same style…
Ctrl-Click (or something similar) and on a participants name and choose:
• Show only utterances with ‘keyword(s)’
• Show only utterances without ‘keyword(s)’
• Expand this section to show more of what was said before or after (either as a command or as an action somehow)
• Copy as Citation (copies this text with citation information to enter into any document, where the citation includes a link to the audio snippet of this text)
• Is there any video from this? (searches timeline for video of this moment in time which was taken near the same location and which the user has access rights to
Liquid interactions where the user can search based on any text, look up references, translate and so on. On OS X this comes for free, through Liquid.
Note: As the user continues this interaction, the amount of data on the screen changes, these are only view changes, but the views should be tracked so the views can be saved and shared.
Create Saved Views
I’m not sure about the language here, whether this should be considered a Report, a Saved View, an Annotation or new Version, but the user should be able to apply considerable control over what it shown in a Channel or across all the users’ accessible media, and store this view and share it with others.
Share Real-Time Coded Documents
Another key feature is the ability to share a stream document and have the real-world time encoded so that anyone can open it and go to not just a specified amount of time in the record but to a specific real-world time, such as 2:37pm. This would entail matching real-world time to the record internal timecode and leaving it in the document itself, maybe in the EXIF data, though this should not stop the record to be opened in a regular media browser.
• User emails a stream document, which looks like a normal media document and someone who is not using any Time Browser software can still open it and play it as normal.
• User emails a stream document, which looks like a normal media document and someone who is using Time Browser software can point to specific real-world time in the player timeline.