Self awareness in hypermusic
0 likes216 views
This document discusses self-awareness in the context of hypermusic nodes. It defines self-awareness and self-expression for nodes. It then describes a solojam application where hypermusic nodes coordinate solos through auctions based on a utility function. Future work is outlined to improve the auction mechanism, user interaction, and test with more nodes including humans and AI.
![feedback loop
Human
Device
Node + Sync +
Node input
input
feedback loop neighbo time of
ur day
Self-awareness
proximit
touch gait
y
Proprio- Music
motion ception orientation temperature Engine
Node
face accel. [x,y,z] RFID
Self-expression
pulse held? direction](https://image.slidesharecdn.com/self-awareness-in-hypermusic-120131054012-phpapp01/85/Self-awareness-in-hypermusic-11-320.jpg)
Self awareness in hypermusic
- 1. self-awareness in hypermusic Arjun Chandra Arve Voldsund Kristian Nymoen Jim Tørresen Kyrre Glette
- 2. outline hypermusic self-awareness hypermusic node solojam future work
- 3. hypermusic Interacting with flexible musical compositions
- 5. Flexible Score Interaction with structures Interpret a structure Detailed Score
- 6. self-awareness as defined by the EPiCS project
- 7. A self-aware node may have: • Private self-awareness • Information about internal state • Public self-awareness • Knowledge about the environment to determine how it is perceived by other parts of the system
- 8. A node exhibits self-expression if it: • Is able to assert its behaviour upon itself or other nodes • And this behaviour is based upon the node's state, context, goals, values, objectives and constraints Furthermore, a node is called proprioceptive if it exhibits both self-awareness and self-expression
- 10. • A hypermusic node is a combination of a human user and the device or instrument (including hardware and software) enabling participation in a hypermusic score. • One could also replace the human with an AI. • This change could be dynamic, i.e. a human could start a node and then move on to another node, while the first continues autonomously from where the human left off. • The human is not required to have a high level of musical proficiency, and the hypermusic node would ideally adapt to the skills of the human.
- 11. feedback loop Human Device Node + Sync + Node input input feedback loop neighbo time of ur day Self-awareness proximit touch gait y Proprio- Music motion ception orientation temperature Engine Node face accel. [x,y,z] RFID Self-expression pulse held? direction
- 12. feedback loop feedback loop Internal Selfexpr Proprio- ception Monitor / Controller External Selfexpr
- 15. Possible hypermusic scenarios / topologies: • Human only nodes • AI only nodes • Mixed human / AI nodes • Small groups (trio, quartet, …) • Medium groups (orchestra, smaller audience at a concert or exhibition) • Large groups (festival or rave party participants) • Local sound generation • Centralised sound generation • Run-time changes between the above configurations
- 16. solojam A band with (artificial) solo artists who co-ordinate their respective solos via auctions
- 17. Main idea: Band with solo artists where: • The artists send bids to the current solo player/leader, in order to become the next leader and play their respective solo. • The current leader plays its respective solo until it can gain from handing over the lead to someone else. • Imagine handing over the playing of solos from one artist to another over time. • Related to a smart camera application in EPiCS
- 18. 10001000 Node • leader 00011100 10101010 Node Node
- 19. Auction • Auction broadcasted by leader at every time step • Auction/Bidding scheme: second price auction • Amounts paid used for book-keeping (“social welfare”)
- 20. Utility function rationale • A player should “understand” what the leader is doing musically, and propose a variation of this • Musically neither too close nor too far away • The current leader should not keep on playing the same solo/pattern for a very long time (fatigue, boredom, …)
- 21. Utility function = hamming distance to leader’s solo (zero as a leader) = duration of play as leader (zero as bidder) a = weighting for = 1.0 b = weighting for = 0.1 c = normalization constant = 2
- 22. Special cases: • If a bidder bids to play a tune with , its utility is assigned to be negative (i.e. it is penalized for wanting to play exactly the same as the leader). • if a leader has just handed over the playing of the solo to another node, it is not allowed to enter the immediate future auction with the same tune as before. This is presently done by assigning it a negative utility. • These cases should be incorporated in a more general way in the future
- 23. Solo encoding and search: • Bit string where each bit represents a fixed position in the beat pattern • Bits are mapped to different drums based on position and player • Searching for a new solo by mutations or random search • current solo is replaced if the new solo increases the utility in the bid
- 24. demo
- 25. Relation to self-awareness definitions • Private self-awareness: information about internal state in terms of knowing what the node wants to play and how long it has been played • Public self-awareness: capable of evaluating their internal state with respect to the environment by way of being able to assign a utility to their respective solos, which takes into account the leader’s solo (public knowledge) and the self-proposed solo (private knowledge)
- 26. Relation to self-awareness definitions • Self-expression (proposing solos, playing sound) based upon • State and context (internal solo, leader/not) • Goals, values, constraints (musical variation, tempo, instruments) • Network topology currently “all-to-all” (may change in future) • Node interaction mechanism can be described as self-interested (“Can we make a mutually beneficial exchange?")
- 27. Discussion • Current utility function is such that it encourages the next leader to play something like the current leader, thus coming up with some kind of synchronisation in a decentralised fashion or at least a smooth transition from one solo to the next. • Social welfare? Calculated as the sum across the instantaneous utilities of each player but not sure what it could mean for the band right now.
- 28. future work solojam and hypermusic in general
- 29. Future work • Ant colony-inspired multicasting approach • Build pheromone trails/graph based on handover dynamics • Use this for communicating auctions • Could be useful with higher number of nodes • Utility function with more interesting dynamics • Improved variation/similarity measures • Base on musical knowledge • Rule-based or data-based?
- 30. Future work • Custom hardware controller • Introduce user interaction • Rhythm input (motion capture or other sensing) • User proficiency measurement from machine learning • Adapt control level to user proficiency • Extend to several layers of musical composition • Drums, bass, melody, … • Several auctions, nodes can bid for participation in different layers
- 31. Future work • Increase the number of nodes, experiments with humans as well as AI nodes • Apply other algorithms and investigate other kinds of dynamics • Test on benchmarks • Bring information back from the music application to the theoretical foundation of the EPiCS project and the other applications
- 32. Testing: selected requirements • Response time (UI and musical) • Priority based processing in node • User-friendliness (intuitive and responsive) • Priority-based communication • Sensor management / tradeoffs (monitor/controller)
- 33. thank you! questions, comments and suggestions are welcome