Monday, August 17, 2015

HetimaNet library support dart:io sockets.

HetimaNet library for dart developer support dart:io socket.  HetimaNet had supported chrome application extension sockets only.  


HetimaNet have following function
  • UDP
  • TCP
  • Http Server
  • Http Client
  • UPnP Portmap 



 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
import 'package:hetimanet/hetimanet.dart';
import 'package:hetimanet/hetimanet_dartio.dart';
//
//
main()  async {
  HetiSocketBuilderDartIO builder = new HetiSocketBuilderDartIO(); 
  UpnpPortMapHelper helper = new UpnpPortMapHelper(builder, "test");
  //
  // get network interface
  List<HetiNetworkInterface> interfaces = await builder.getNetworkInterfaces();
  for (HetiNetworkInterface i in interfaces) {
    print("<ni>${i.address} ${i.prefixLength} ${i.name}");
  }
  //
  // portmapping 
  try {
    StartGetExternalIp exip = await helper.startGetExternalIp(reuseRouter: true);
    print("<exip> ${exip.externalIp}");
  } catch (e) {
    print("<exip ERROR> ${e}");
  }
  //
  // get local ip
  try {
    StartGetLocalIPResult loip = await helper.startGetLocalIp();
    for(HetiNetworkInterface i in loip.networkInterface) {
      print("<glip> ${i.address} ${i.name}");      
    }
  } catch (e) {
    print("<glip ERROR> ${e}");
  }
  //
  // start portmap
  try {
    StartPortMapResult sp = await helper.startPortMap();
    print("<add> ${sp}");
  } catch (e) {
    print("<add ERROR> ${e}");
  }
  //
  // end portmap
  try {
    DeleteAllPortMapResult ep = await helper.deletePortMapFromAppIdDesc();
    print("<del> ${ep}");
  } catch (e) {
    print("<del ERROR> ${e}");
  }
}

Source

https://github.com/kyorohiro/dart_hetimanet
https://github.com/kyorohiro/dart_hetimanet/tree/master/example/HetimaPortMap

Port map chrome app

- Chrome Store

https://chrome.google.com/webstore/detail/hetimaportmap/naifildeohmcocnmibaampijofhcohif

--------------
kyorohiro work



Friday, August 14, 2015

Let's coding mainline dht.(6) End

Can find value to use get_peers message


  • use get_peers message to find value
  • use announce_peer message  to save value in p2p network


We could create p2p network in previous section. I explain to save value in p2p network.

get_peers message and find_node are almost similar.


MainLine DHT use get_peers message to set value in p2p network. get_peers message have two role.

  • when receive get_peers query, if knode have request value, knode  send response message about value.
  • if knode dont have value, knode send reposese message about close peer from kid.

Knode repose to send get_peers message, then, knode can find close peer, and value.

Those system and find_nodes is almost simillar.

  •  introduce close peer form KID
  •  repeate to send meesage 




save data to use announce_peer message

When Knode find k-number of close knode to repeat sending get_peers message. Knode send announce_peer message to save value.

Knode send announce_peer message into k-number node.
A robustness rises by requesting value into multiple node.


  • It is easy to find valie
  • It is safe to disconnect node
  • It's possible to disperse a load.


message struct


```
arguments:
{
  "t":"aa",
  "y":"q",
  "q":"get_peers",
  "a": {
    "id" : "<querying nodes id>",
    "info_hash" : "<20-byte infohash of target torrent>"
  }
}

response: have value
{
  "id" : "<queried nodes id>",
  "token" :"<opaque write token>",
  "values" :
  ["<peer 1 info string>", "<peer 2 info string>"]
}

response: have node info
{
 "t":"aa",
 "y":"r",
 "r": {
  "id" : "<queried nodes id>",
  "token" :"<opaque write token>",
  "nodes" : "<compact node info>"
  }
}
```

"token" and "values" is  appeared for the first time int this blog.
"token" is byte date that responser define this byte data and byte data length. This value ise used to send announce_peers query.
"value" is list that contain 6bytes data[<IP 4bute> ,<Port 2byte>]. this values is peer ip and port that's peer can share file to use torrent client bep003.

ref http://www.bittorrent.org/beps/bep_0005.html


About Implements 


get_peers implements is almost similar as find_node. We can code get_peers to use find_node implements 's know-how.



Test DHT


  •  test dht

You'll make them move actually. I explain mainline dht sample to use hetimatorrent library.

find value from special kid  int p2p network.


```

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
main() {
  KNode node = new KNode(new HetiSocketBuilderChrome(), verbose: true);

  //
  node.start(ip: "0.0.0.0", port: 28080).then((_) {
    node.onGetPeerValue.listen((KGetPeerValue v) {
      print("---onGetPeerValue ${v.ipAsString} ${v.port} ${v.infoHashAsString} ");
    });
  });

  // initial node
  String initailNodeIp = "0.0.0.0";
  int initailNodePort = 38080;
  node.addBootNode(initailNodeIp, initailNodePort);
  
  // search target
  List<int> infoHash = new List.filled(20, 4);
  node.startSearchValue(new KId(infoHash), 18080, getPeerOnly: true);
  
  new Future.delayed(new Duration(minutes:30)).then((_){
    print(node.rootingtable.toInfo());
    return node.stop();
  });

}
```


Sample app with gui.

https://github.com/kyorohiro/dart_hetimatorrent/tree/master/example/TorrentDHT




Ref


http://www.bittorrent.org/beps/bep_0005.html
http://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf


PS

GitBook nazenani Torrent for japanese
- https://www.gitbook.com/book/kyorohiro/doc_hetimatorrent/details
hetima torrent library
- https://github.com/kyorohiro/dart_hetimatorrent
- https://github.com/kyorohiro/dart_hetimatorrent/tree/master/example/TorrentDHT


-------
Kyorohiro work

http://kyorohiro.strikingly.com

Thursday, August 13, 2015

Let's coding mainline dht.(5) Implements FindNodes

Implements FindNodes

(1) Add findnode function in RootingTable.


First. I add findnode function in rootingtable class.  This function can search close peer info from  target KID to sort KIDs .

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
class KRootingTable {
  ...
  ...
  ...
  List<KPeerInfo> findNode(KId id) {
    List<KPeerInfo> ids = [];
    for (KBucket b in _kBuckets) {
      for (KPeerInfo i in b.iterable) {
        ids.add(i);
      }
    }
    ids.sort((KPeerInfo a, KPeerInfo b) {
      return a.id.xor(id).compareTo(b.id.xor(id));
    });
    List<KPeerInfo> ret = [];
     for (KPeerInfo p in ids) {
      ret.add(p);
       if (ret.length >= _kBucketSize) {
         return ret;
      }
    }
    return ret;
  }
  ...
  ...
}

(1) KNode have UDP server function

Knode use udp socket to communication between node and node. and use bencode. This term add binding to use udp server and parse message.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
class KNode {
  bool _isStart = false;
  bool get isStart => _isStart;
  HetiSocketBuilder _socketBuilder = null;
  HetiUdpSocket _udpSocket = null;

  KNode(HetiSocketBuilder socketBuilder) {
    this._socketBuilder = socketBuilder;
  }
 
  Future start({String ip: "0.0.0.0", int port: 28080}) async {
    (_isStart != false ? throw "already started" : 0);
    _udpSocket = this._socketBuilder.createUdpClient();
    return _udpSocket.bind(ip, port, multicast: true).then((int v) {
      _udpSocket.onReceive().listen((HetiReceiveUdpInfo info) {
        KrpcMessage.decode(info.data, this).then((KrpcMessage message) {
          onReceiveMessage(info, message);
        });
      });
      _isStart = true;
    });
  }

  Future stop() async {
    if (_isStart == false || _udpSocket == null) {
      return null;
    }
    return _udpSocket.close().whenComplete(() {
      _isStart = false;
      _ai.stop(this);
    });
  }
}

(2) KNode have Krpc Message parse function.


MainLine DHT use KRPC protocol with bencode. this term create to prase krpc message . but We had already create bencod barser. It is easy.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
class KrpcMessage {
  KrpcMessage.fromMap(Map map) {
    _messageAsMap = map;
  }

  static Future<KrpcMessage> decode(List<int> data) async {
    Map<String, Object> messageAsMap = null;
    try {
      Object v = Bencode.decode(data);
      messageAsMap = v;
    } catch (e) {
      throw {};
    }
    return  new KrpcMessage.fromMap(messageAsMap);
  }
}


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
class KrpcMessage {
...
...
  List<int> get transactionId => _messageAsMap["t"]);
  String get transactionIdAsString => UTF8.decode(transactionId);

  //
  List<int> get messageType => _messageAsMap["y"];
  String get messageTypeAsString => UTF8.decode(messageType);

  //
  List<int> get query => _messageAsMap["q"];
...
...
}
````


(3) KNode have seding message function


```

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
class KNode {
 ..
 ..
  sendMessage(KrpcMessage message, String ip, int port) {
      return _udpSocket.send(message.messageAsBencode, ip, port);
  }
 ..
}

class FindNode {

  static int queryID = 0;

  static KrpcMessage createQuery(List<int> queryingNodesId, List<int> targetNodeId) {
    List<int> transactionId = UTF8.encode("fi${queryID++}");
    return new KrpcMessage.fromMap({"a": {"id": queryingNodesId, "target": targetNodeId}, "q": "find_node", "t": transactionId, "y": "q"});
  }

  static KrpcMessage createResponse(List<int> compactNodeInfo, List<int> queryingNodesId, List<int> transactionId) {
    return new KrpcMessage.fromMap({"r": {"id": queryingNodesId, "nodes": compactNodeInfo}, "t": transactionId, "y": "r"});
  }
}
```

When send message, KNode use binded UDPSocket. Received message node can know port and ip  to use  socket to receive message.

(4) KNode have join network function


KNode send findnode query that k-number close peer in rootingTable.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
class KNodeWorkFindNode {
  ...
  ...
  updateP2PNetworkWithoutClear(KNode node) {
    node.rootingtable.findNode(node.nodeId).then((List<KPeerInfo> infos) {
      for (KPeerInfo info in infos) {
        if (!_findNodesInfo.rawsequential.contains(info)) {
          _findNodesInfo.addLast(info);
          node.sendFindNodeQuery(info.ipAsString, info.port, node.nodeId.value).catchError((_) {});
        }
      }
    });
  }
  ...
  ...
}

when receive response, then KNode send findnode query again.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
class KNodeWorkFindNode {
  ...
  ...
  onReceiveQuery(KNode node, HetiReceiveUdpInfo info, KrpcMessage query) {
    if (query.queryAsString == KrpcMessage.QUERY_FIND_NODE) {
      KrpcFindNode findNode = query.toFindNode();
      return node.rootingtable.findNode(findNode.targetAsKId).then((List<KPeerInfo> infos) {
        return node.sendFindNodeResponse(info.remoteAddress, info.remotePort, query.transactionId, KPeerInfo.toCompactNodeInfos(infos)).catchError((_) {});
      });
    }
    node.rootingtable.update(new KPeerInfo(info.remoteAddress, info.remotePort, query.nodeIdAsKId));
    updateP2PNetworkWithoutClear(node);
  }
  ...
  ..
}

and If you were a certain period of time, to access it again


1
2
3
4
5
6
7
8
class KNodeWorkFindNode {
  ...
  ...

  onTicket(KNode node) {
    _findNodesInfo.clear();
    updateP2PNetworkWithoutClear(node);
  }


(5) When receive FindeNode query, KNode send findnode response


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
class KNodeWorkFindNode {
  ...
  ...
  onReceiveResponse(KNode node, HetiReceiveUdpInfo info, KrpcMessage response) {
    if (response.queryFromTransactionId == KrpcMessage.QUERY_FIND_NODE) {
      KrpcFindNode findNode = response.toFindNode();
      for (KPeerInfo info in findNode.compactNodeInfoAsKPeerInfo) {
        node.rootingtable.update(info);
      }
    }
    node.rootingtable.update(new KPeerInfo(info.remoteAddress, info.remotePort, response.nodeIdAsKId));
    updateP2PNetworkWithoutClear(node);
  }
  ..
}


Next

About GetPeers

Ref

http://www.bittorrent.org/beps/bep_0005.html
http://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf

PS

- GitBook Nazenani Torrent (for japanese)
https://www.gitbook.com/book/kyorohiro/doc_hetimatorrent/details

-source
  https://github.com/kyorohiro/dart_hetimatorrent
 https://github.com/kyorohiro/dart_hetimatorrent/tree/master/example/TorrentDHT

-------
Kyorohiro work



Tuesday, August 11, 2015

About Nazenani Torrent

I am writing ebook about The know-how gained during the Torrent Client development.

- Nazenani Torrent (for japanese only now)
https://www.gitbook.com/book/kyorohiro/doc_hetimatorrent/details

-------
Kyorohiro work

Let's coding mainline dht.(4) build p2p network at findNodes query

My English skill is poor. If I make mistakes in my English,please pardon me.

Build p2p betwork at FindNode query

  • Build p2p network to use only FindNodes query
  • Update routingtable to receive a response.
I explained about kBucket Rooting table in previous section.  In this section, We build p2p network to update rooting table.


Could build network by findnodes qurey

Mainline DHT(Kademlia) use UDP socket to communicatio other peer.
P2P network is building to use only findnodes qurey.

Could find close distance peer from the specified KID

We use FindNode query,  We could find close distance peer from the specified KID.

findnodes query and response have following structure.
 arguments:  
 {  
  "t":"aa",  
  "y":"q",  
  "q":"find_node",  
  "a":{  
  "id" : "<nodes id>",  
  "target" : "<id of target node>"  
  }  
 }  
 response:  
 {  
  "t":"aa",  
  "y":"r",  
  "r":{  
  "id" : "<nodes id>",  
  "nodes" : "<compact node info>"  
  }  
 }  

"t" is specified string. that any sender can be determined.this value use when receive response.

"id" is sender peer's peer Id
"target" findnode querry return peers that the distance is close.
"nodes" is 26byte byte data list. this data have 20byte Peer ID, and IP, and port,

Join/Build p2p network.

It is updating rooting table to use findnodes query that "id" is own peer id.
repeat to request findnodes query until k-number of close peer is no longer update.

following table is a peer 's rooting table on simulator use 1000 peer.
Rooting Table indexnumner of Peercomment
01own
1471..
1541..
1554..
1564..
1576..
1588..
1598..
1608..

Become same result If try this in torrent 's dht network .

147 is very close node.

Ref


- GitBook Nazenani Torrent for japanese


- kyorohiro work torrent library and dht demo


-------
Kyorohiro work

Let's coding mainline dht.(3) implements kBucket RootingTable

My English skill is poor. If I make mistakes in my English,please pardon me.
I will make description and implementation about Mainline DHT.

Let's coding RootingTable

  • Implements KId
  • Implements kBucket
  • Implements RootingTable
Let's coding RootingTable,

Implements KID

express  InfoHash and PeerID as KID.

We must tot calc xor distance of InfoHash and InfoHash, Peer ID and InfoHash, Peer ID and Peer. Those ID can define 20 byte array.  It's call KID in sentense.

- It isn't necessary to change KID into the numerical value.

KID is 160bit valiue. but, dartlang support 53bit only. SO I have usually implemented biginteger. but It isn't necessary to change KID into the numerical value to use in kbucket rooting table.

First.  First definition, KID has 20 byte array

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
class KId {
  List<int> _values = null;
  List<int> get value => new List.from(_values);
  KId(List<int> id) {
    if(id == null || id.length != 20) {
      throw {};
    }
    this._values = new Uint8List.fromList(id);
  }
  int get length => _values.length;
  int operator [](int idx) => _values[idx];
  Iterator<int> get iterator => _values.iterator;
}

- add  func that calc xor 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
class KId {
  ...
  ...
  ...
  KId xor(KId b, [KId output = null]) {
    if (output == null) {
      output = new KId.zeroClear();
    }
    for (int i = 0; i < b._values.length; i++) {
      output._values[i] = this._values[i] ^ b._values[i];
    }
    return output;
  }
}

- add func that big small comparison

We must to add big and small comparison function to sort KID in rooting table.
We has become possible to find close peer in  KID list by sorting function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
class KId {
  ...
  ...
  ...
  bool operator >(KId b) {
    for (int i = 0; i < b._values.length; i++) {
      if (this._values[i] == b._values[i]) {
        continue;
      } else if (this._values[i] > b._values[i]) {
        return true;
      } else {
        return false;
      }
    }
    return false;
  }

  bool operator ==(KId b) {
    for (int i = 0; i < b._values.length; i++) {
      if (this._values[i] != b._values[i]) {
        return false;
      }
    }
    return true;
  }

  bool operator >=(KId b) {
    return (this == b ? true : (this > b ? true : false));
  }

  bool operator <(KId b) {
    return (this == b ? false : !(this > b));
  }

  bool operator <=(KId b) {
    return (this == b ? true : (this > b ? false : true));
  }
}

https://github.com/kyorohiro/dart_hetimatorrent/tree/master/lib/src/dht


Implements kBucket

kBucket is container that can contain K-number of peer info. We can implements as list with limit on when you want to add value.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
class KBucket {
  int _k = 8;
  int get k => _k;
  List<KPeerInfo> peerInfos = null;

  KBucket(int kBucketSize) {
    this._k = kBucketSize;
    this.peerInfos = [];
  }

  add(KPeerInfo peerInfo) {
    if (peerInfos.contains(peerInfo) == true) {
      peerInfos.remove(peerInfo);
    }
    peerInfos.add(peerInfo);
    if (peerInfos.length > k) {
      peerInfos.removeAt(0);
    }
  }

  int get length => peerInfos.length;
  KPeerInfo operator [](int idx) => peerInfos[idx];
  Iterator<KPeerInfo> get iterator => peerInfos.iterator;
}

https://github.com/kyorohiro/dart_hetimatorrent/tree/master/lib/src/dht


Implements RootingTable

Rooting table has 161- number of kBucket.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
class KRootingTable {
  List<KBucket> _kBuckets = [];
  int _kBucketSize = 0;

  KRootingTable(int k_bucketSize) {
    this._kBucketSize = k_bucketSize;
    for (int i = 0; i < 161; i++) {
      _kBuckets.add(new KBucket(k_bucketSize));
    }
  }
}


Implements calc rooting table index.

I explain kid into the numerical value to calc rooting table index in previous section.

2進index
00001
2, 3010, 0112
4, 5, 6, 7 100, 101, 110, 1113

And went to check one by 1bit from left to right, by the first place was a 1, you will see that kBucket position is determined.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
class KRootingTable {
  List<KBucket> _kBuckets = [];
  int _kBucketSize = 0;
  KId _ownerKId = null;
  KId get ownerKId => _ownerKId;

  KRootingTable(int k_bucketSize, KId ownerKId) {
    this._kBucketSize = k_bucketSize;
    for (int i = 0; i < 161; i++) {
      _kBuckets.add(new KBucket(k_bucketSize));
    }
    this._ownerKId = ownerKId;
  }

  int getRootingTabkeIndex(KId v) {
    // calc xor
    v = v.xor(_ownerKId);

    // clac index
    for (int i = 0, ret = 19; i < 20; i++, ret--) {
      if (v[i] != 0) {
        for (int j = 0; j < 9; j++) {
          if (v[i] < (0x1 << j)) {
            return (ret * 8) + j;
          }
        }
        return i;
      }
    }
    return 0;
  }
}

Add peer info in kBucket.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
class KRootingTable {
  ...
  ...
  ...
  Future update(KPeerInfo info) {
    return new Future(() {
      _kBuckets[getRootingTabkeIndex(info.id)].add(info);
    });
  }
  ...
  ...
}

Now, the creation of RootingTable is complete.  I will explain findNode in next section. and append fincNode function in RootingTable class.

Ref

http://www.bittorrent.org/beps/bep_0005.html
http://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf

- GitBook Nazenani Torrent for japanese

https://www.gitbook.com/book/kyorohiro/doc_hetimatorrent/details

- kyorohiro work torrent library and dht demo

 https://github.com/kyorohiro/dart_hetimatorrent
 https://github.com/kyorohiro/dart_hetimatorrent/tree/master/example/TorrentDHT

-------
Kyorohiro work

http://kyorohiro.strikingly.com

Monday, August 10, 2015

Let's coding mainline dht.(2) About kBucket RootingTable

My English skill is poor. If I make mistakes in my English,please pardon me.
I will make description and implementation about Mainline DHT.

MainlineDHTはKademuliaのkBucketを利用している

  • Mainline DHT use Kademulia
  • Mainline DHT distance is XOR
  • Mainline DHT have kBucket RootingTable

Torrent support DHT is Kademulia.(http://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf)
I explain kBucket and rooting table in this section.

Mainline DHT use XOR distance

 I exlaine that DHT must to define distance in previous section. Kademulia use xor distance as DHT distance definition.
Do you know how about xor distance? Would you try check with me about xor distance!!

- It cab be express number as binary

Number can be express as binary in computer science.  Binary have only 1 and 0. for example, 256 can express as 11111111 , 1 can express as 00000001, 2 ca express as 00000010, 3 can expressed as 00000011 and 5 can express as 0000101.

- calculate XOR after converting binary

  We can get xor distance to calculate xor after converting binary. and I write as A ^ B , to be calulate xor A and B .

First, what is xor? xor is logical operation. that is outputs true only when both inputs differ,

Value AValue B A ^ B 
110
101
011
110

This logical operation execute about each digit.
"256(11111111) ^ 3(00000011) --> 252 (11111100)"and "252 (11111100) ^ 256(11111111) --> 3(00000011)"

XOR is symmetric distance 

XOR is symmetric distance,  It is same value A ^ B  and B ^ A .
For example. "256(11111111) ^ 3(00000011) --> 252 (11111100)"" 3(00000011) ^ 256(11111111) --> 252 (11111100)".

KBucket and  RootingTable come together, It will appear excellent features.


MainLine DHT use kBucket's RootingTable

I explain DHT have peer list in previous section.  and peer list have bias.   Kademulia use kBucket RootingTable to realize those characteristic.

kBucket is container that  K of peer, grouping. Only that. kBucket cound not cotain more than  K.  if cotain more peer info, We must to remove peer info from kBucket.

Kademulia 's RootingTable can possess K of kBucket, 0-160.  those kBucket can possess,  smaller than 0st power of 2 ,  1st power of 2,  and Kst power of 2.

It's slightly complicated, isn't it? I'll see specifically.

The intuitive structure to be convert a figure.

First,  Would you check xor distance!! but , this part use 3bit ID. Because it's promotion to handle  160bit (20byte).

Value BinaryA ^ 010table index
000022
100132
201000
301111
410063
510173
611043
711153

as above table. I wrote about a peer which's id is 010.
for example, In case of value is 5, Binary is 101, xor distance is 7, and kBucket's number is 3.



Next , show figure. How is it?  Isn't it intuitive arrangement? Every time a branch moves, the value of index becomes big.


PS

----------
Kyorohiro work

Let's coding mainline dht with dartlang (1)

My English skill is poor. If I make mistakes in my English,please pardon me.

I will make description and implementation about Mainline DHT. 

Mainline DHT can search peer without tracker server

  • It is not necessary to use tracker server
  • The concept of six degrees of separation
  • Define the distance for building a p2p

Can build p2p network without tracker server

  Torrent client had to use tracker server to find peer in bep003. but, The Mainline DHT realized the role of this tracker server on p2p network only.
   We needed to prepare tracker server on the world wide web. but, DHT changed this scenario. We could share data to use only p2p network by DHT.

Use the concept of six degrees of separation

 How can we create p2p network that is substitute for tracker server?
Concretely speaking,  we must to realize  p2p network that return response has special peer list when receive InfoHash.

- It is realized to repeat request for introducing well informed person.

Do you know about "The concept of use six degrees of separation"?  I think that DHT has same concept.

 The theory that everyone and everything is six or fewer steps away, by way of introduction, from any other person in the world, so that a chain of "a friend of a friend" statements can be made to connect any two people in a maximum of six steps.(https://en.wikipedia.org/wiki/Six_degrees_of_separation)

We can send message into some person to repeat request about well informed person.

- p2p network use same rule.

p2p network use the same rule too. repeat to request messae.

  • if a peer has peers for the InfoHash, return peer information
  • if a peer don't has peer for the InfoHash, return closet peer information to InfoHash.


Must to define distance

 A little more specifically, We must to think about how to find closed peer in peers. We can solve this problem to define distance for a peer and InfoHash.   If we can define distance,  We can find close peer to calculates distance in peers.


Must to create biased information on p2p network

And we must to bias stocked peers  to realize the six degrees of separation.

There is bias about  our list of acquaintance.  We are well-known about our family, our colleague and our friends of same club. but We are not well-known about person of another company, and person of another country.

If this p2p network don't have bias like this. It is not possible to find peer even if introduce clost peers at infohash.

 A peer must to be well-known about close peer.  and peer must to be not well-known about distant peer.

-------
Kyorohiro work

http://kyorohiro.strikingly.com

Sunday, August 9, 2015

WegGL Trial with Dartlang fragment shader test

fragment shader test. draw circle at mouse position.


  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
import 'dart:html' as html;
import 'dart:web_gl' as webgl;
import 'dart:typed_data' as data;
import 'dart:async';

void main() {
  var canvas = new html.CanvasElement(width: 500, height: 500);
  html.document.body.append(canvas);
  double x = 0.0;
  double y = 0.0;
  canvas.onMouseMove.listen((html.MouseEvent e) {
    html.Rectangle t =  canvas.getBoundingClientRect();
    x = e.clientX - t.left;
    y = e.clientY - t.top;
  });

  webgl.RenderingContext GL = canvas.getContext3d();

  // setup shader
  webgl.Shader vertexShader = loadShader(GL,
      webgl.RenderingContext.VERTEX_SHADER,
          "attribute vec3 vp;\n" +
          "void main() {\n" +
          "  gl_Position = vec4(vp, 1.0);\n" +
          "}\n");

  webgl.Shader fragmentShader = loadShader(GL,
      webgl.RenderingContext.FRAGMENT_SHADER, 
      "precision mediump float;\n" +
      "uniform float t;\n" +
      "uniform float x;\n" +
      "uniform float y;\n" +
      "void main() {\n" +
      " float r = (cos(t)+1.0)/2.0;\n"
      " float a = (sin(t)+1.0)/2.0;\n"
      " float gb = 100.0-sqrt((gl_FragCoord.x-x)*(gl_FragCoord.x-x)+(gl_FragCoord.y-y)*(gl_FragCoord.y-y));\n"
      " gl_FragColor = vec4(r, gb/100.0, gb/100.0, a);\n" +
     "}\n");

  webgl.Program shaderProgram = GL.createProgram();
  GL.attachShader(shaderProgram, fragmentShader);
  GL.attachShader(shaderProgram, vertexShader);
  GL.linkProgram(shaderProgram);
  GL.useProgram(shaderProgram);

  if (false ==
      GL.getProgramParameter(
          shaderProgram, webgl.RenderingContext.LINK_STATUS)) {
    String message = "alert: Failed to linked shader";
    throw new Exception("${message}\n");
  }

  //
  // setup
  // leftup (x, y, z), leftdown, rightup, rightdown
  data.TypedData rectData = new data.Float32List.fromList(
      [-0.8, 0.8, 0.0, -0.8, -0.8, 0.0, 0.8, 0.8, 0.0, 0.8, -0.8, 0.0]);
  data.TypedData rectDataIndex =
      new data.Uint16List.fromList([0, 1, 2, 1, 3, 2]);

  webgl.Buffer rectBuffer = GL.createBuffer();
  GL.bindBuffer(webgl.RenderingContext.ARRAY_BUFFER, rectBuffer);
  GL.bufferData(
      webgl.ARRAY_BUFFER, rectData, webgl.RenderingContext.STATIC_DRAW);

  webgl.Buffer rectIndexBuffer = GL.createBuffer();
  GL.bindBuffer(webgl.ELEMENT_ARRAY_BUFFER, rectIndexBuffer);
  GL.bufferDataTyped(webgl.RenderingContext.ELEMENT_ARRAY_BUFFER, rectDataIndex,
      webgl.RenderingContext.STATIC_DRAW);

  //
  // draw

  int startTime = new DateTime.now().millisecondsSinceEpoch;
  int count = 0;
  render(_) {
    count++;
    GL.clearColor(0.6, 0.2, 0.2, 1.0);
    GL.clear(webgl.RenderingContext.COLOR_BUFFER_BIT);

    int locationVertexPosition = GL.getAttribLocation(shaderProgram, "vp");
    GL.vertexAttribPointer(
        locationVertexPosition, 3, webgl.RenderingContext.FLOAT, false, 0, 0);
    GL.enableVertexAttribArray(locationVertexPosition);

    int lastTime = new DateTime.now().millisecondsSinceEpoch;
    double t = (lastTime - startTime) *0.001;
    //print("##${t}");

    webgl.UniformLocation timeLocation = GL.getUniformLocation(shaderProgram, "t");
    GL.uniform1f(timeLocation, t);
    
    webgl.UniformLocation mouseX = GL.getUniformLocation(shaderProgram, "x");
    webgl.UniformLocation mouseY = GL.getUniformLocation(shaderProgram, "y");
    GL.uniform1f(mouseX, x);
    GL.uniform1f(mouseY, 500-y);

    GL.drawElements(webgl.RenderingContext.TRIANGLES, 6,
        webgl.RenderingContext.UNSIGNED_SHORT, 0);

    new Future.delayed(new Duration(milliseconds:10)).then((render));
    if(count %100 == 0) {
      print("fps=${count/t.toInt()};x=${x};y=${y}");
    }
  }

  render(0);
}

webgl.Shader loadShader(webgl.RenderingContext context, int type, var src) {
  webgl.Shader shader = context.createShader(type);
  context.shaderSource(shader, src);
  context.compileShader(shader);
  if (false ==
      context.getShaderParameter(
          shader, webgl.RenderingContext.COMPILE_STATUS)) {
    String message =
        "Error compiling shader ${context.getShaderInfoLog(shader)}";
    context.deleteShader(shader);
    throw new Exception("${message}\n");
  }
  return shader;
}
>
-------
Kyorohiro work

kyorohiro.strikingly.com


Saturday, August 8, 2015

HetimaTorrentClient ver 0.1.0 implements DHT with Pure Dartlang


HetimaTorrentClient corresponded to DHT at  ver 0.1.0.


This is Pre-Alpha instability version now.

This application was a test to share Big Buck Bunny movie with 1GB over DHT. 

[DHT]
Torrent client use tracker server to find other client joined p2p network.  but, if tracker server is died , We could not find other client. but, DHT supported torrent client could find it. DHT support torrent client  could make p2p network as an alternative to tracker server. 

- bep5 

Thursday, August 6, 2015

WegGL Trial with Dartlang draw rect

Dartlang has very comfortable programming environment.Dart x Eclipse 's strong auto-complete function is excellent.  So, I have started to study WebGL with dartlang.

create draw rect sample




 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
import 'dart:html' as html;
import 'dart:web_gl' as webgl;
import 'dart:typed_data' as data;

void main() {
  var canvas = new html.CanvasElement(width: 500, height: 500);
  html.document.body.append(canvas);

  webgl.RenderingContext GL = canvas.getContext3d();
  GL.clearColor(0.6, 0.2, 0.2, 1.0);
  GL.clear(webgl.RenderingContext.COLOR_BUFFER_BIT);

  // setup shader
  webgl.Shader vertexShader = loadShader(GL, webgl.RenderingContext.VERTEX_SHADER, 
          "attribute vec3 vp;\n" +
          "void main() {\n" +
          "  gl_Position = vec4(vp, 1.0);\n" +
          "}\n");

  webgl.Shader fragmentShader = loadShader(GL, webgl.RenderingContext.FRAGMENT_SHADER, 
          "precision mediump float;\n" +
          "void main() {\n" +
          " gl_FragColor = vec4(0.0,1.0,0.0,1.0);\n" +
          "}\n");

  webgl.Program shaderProgram = GL.createProgram();
  GL.attachShader(shaderProgram, fragmentShader);
  GL.attachShader(shaderProgram, vertexShader);
  GL.linkProgram(shaderProgram);
  GL.useProgram(shaderProgram);

  if (false == GL.getProgramParameter(shaderProgram, webgl.RenderingContext.LINK_STATUS)) {
    String message = "alert: Failed to linked shader";
    throw new Exception("${message}\n");
  }

  //
  // setup
  // leftup (x, y, z), leftdown, rightup, rightdown
  data.TypedData rectData = new data.Float32List.fromList([-0.8, 0.8, 0.0, -0.8, -0.8, 0.0, 0.8, 0.8, 0.0, 0.8, -0.8, 0.0]);
  data.TypedData rectDataIndex = new data.Uint16List.fromList([0, 1, 2, 1, 3, 2]);

  webgl.Buffer rectBuffer = GL.createBuffer();
  GL.bindBuffer(webgl.RenderingContext.ARRAY_BUFFER, rectBuffer);
  GL.bufferData(webgl.ARRAY_BUFFER, rectData, webgl.RenderingContext.STATIC_DRAW);

  webgl.Buffer rectIndexBuffer = GL.createBuffer();
  GL.bindBuffer(webgl.ELEMENT_ARRAY_BUFFER, rectIndexBuffer);
  GL.bufferDataTyped(webgl.RenderingContext.ELEMENT_ARRAY_BUFFER, rectDataIndex, webgl.RenderingContext.STATIC_DRAW);

  //
  // draw
  int locationVertexPosition = GL.getAttribLocation(shaderProgram, "vp");
  GL.vertexAttribPointer(locationVertexPosition, 3, webgl.RenderingContext.FLOAT, false, 0, 0);
  GL.enableVertexAttribArray(locationVertexPosition);
  GL.drawElements(webgl.RenderingContext.TRIANGLES, 6, webgl.RenderingContext.UNSIGNED_SHORT, 0);
}

webgl.Shader loadShader(webgl.RenderingContext context, int type, var src) {
  webgl.Shader shader = context.createShader(type);
  context.shaderSource(shader, src);
  context.compileShader(shader);
  if (false == context.getShaderParameter(shader, webgl.RenderingContext.COMPILE_STATUS)) {
    String message = "Error compiling shader ${context.getShaderInfoLog(shader)}";
    context.deleteShader(shader);
    throw new Exception("${message}\n");
  }
  return shader;
}

- source
https://github.com/kyorohiro/dart_hetimagl/tree/master/web

-------
Kyorohiro work

kyorohiro.strikingly.com