Thompson sampling for opportunistic spectrum access with Markovian rewards

Suleman Alnatheer; Hong Man

doi:10.1109/VTCFall.2014.6966204

Thompson sampling for opportunistic spectrum access with Markovian rewards

Suleman Alnatheer
, Hong Man

Department of Electrical and Computer Engineering

Stevens Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This paper considers the problem of multi- channels opportunistic access in stationary wireless environments, where an intelligent and unlicensed agent uses the existing licensed spectrum to maximize it's throughput while not causing harm to existing licensed users. Each channel is modeled as unknown and never ending stream of binary data generated by first order discrete time Markov process with two states Gilbert-Elliot model. This problem is classified as restless bandit, where optimal solution is intractable. An on-line approximation algorithm is proposed based on Thompson Sampling Algorithm, which acts as heuristic search for best channel in the spectrum.

Original language	English
Title of host publication	2014 IEEE 80th Vehicular Technology Conference, VTC 2014-Fall - Proceedings
Edition	November
ISBN (Electronic)	9781479944491
DOIs	https://doi.org/10.1109/VTCFall.2014.6966204
State	Published - 24 Nov 2014
Event	80th IEEE Vehicular Technology Conference, VTC 2014-Fall - Vancouver, Canada Duration: 14 Sep 2014 → 17 Sep 2014

Publication series

Name	IEEE Vehicular Technology Conference
Number	November
Volume	2014-November
ISSN (Print)	1090-3038

Conference

Conference	80th IEEE Vehicular Technology Conference, VTC 2014-Fall
Country/Territory	Canada
City	Vancouver
Period	14/09/14 → 17/09/14

Access to Document

10.1109/VTCFall.2014.6966204

Cite this

@inproceedings{95836ab1a02d4cd7bb02b2ef63a77e6a,

title = "Thompson sampling for opportunistic spectrum access with Markovian rewards",

abstract = "This paper considers the problem of multi- channels opportunistic access in stationary wireless environments, where an intelligent and unlicensed agent uses the existing licensed spectrum to maximize it's throughput while not causing harm to existing licensed users. Each channel is modeled as unknown and never ending stream of binary data generated by first order discrete time Markov process with two states Gilbert-Elliot model. This problem is classified as restless bandit, where optimal solution is intractable. An on-line approximation algorithm is proposed based on Thompson Sampling Algorithm, which acts as heuristic search for best channel in the spectrum.",

author = "Suleman Alnatheer and Hong Man",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.; 80th IEEE Vehicular Technology Conference, VTC 2014-Fall ; Conference date: 14-09-2014 Through 17-09-2014",

year = "2014",

month = nov,

day = "24",

doi = "10.1109/VTCFall.2014.6966204",

language = "English",

series = "IEEE Vehicular Technology Conference",

number = "November",

booktitle = "2014 IEEE 80th Vehicular Technology Conference, VTC 2014-Fall - Proceedings",

edition = "November",

}

Alnatheer, S & Man, H 2014, Thompson sampling for opportunistic spectrum access with Markovian rewards. in 2014 IEEE 80th Vehicular Technology Conference, VTC 2014-Fall - Proceedings. November edn, 6966204, IEEE Vehicular Technology Conference, no. November, vol. 2014-November, 80th IEEE Vehicular Technology Conference, VTC 2014-Fall, Vancouver, Canada, 14/09/14. https://doi.org/10.1109/VTCFall.2014.6966204

Thompson sampling for opportunistic spectrum access with Markovian rewards. / Alnatheer, Suleman; Man, Hong.
2014 IEEE 80th Vehicular Technology Conference, VTC 2014-Fall - Proceedings. November. ed. 2014. 6966204 (IEEE Vehicular Technology Conference; Vol. 2014-November, No. November).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Thompson sampling for opportunistic spectrum access with Markovian rewards

AU - Alnatheer, Suleman

AU - Man, Hong

PY - 2014/11/24

Y1 - 2014/11/24

N2 - This paper considers the problem of multi- channels opportunistic access in stationary wireless environments, where an intelligent and unlicensed agent uses the existing licensed spectrum to maximize it's throughput while not causing harm to existing licensed users. Each channel is modeled as unknown and never ending stream of binary data generated by first order discrete time Markov process with two states Gilbert-Elliot model. This problem is classified as restless bandit, where optimal solution is intractable. An on-line approximation algorithm is proposed based on Thompson Sampling Algorithm, which acts as heuristic search for best channel in the spectrum.

AB - This paper considers the problem of multi- channels opportunistic access in stationary wireless environments, where an intelligent and unlicensed agent uses the existing licensed spectrum to maximize it's throughput while not causing harm to existing licensed users. Each channel is modeled as unknown and never ending stream of binary data generated by first order discrete time Markov process with two states Gilbert-Elliot model. This problem is classified as restless bandit, where optimal solution is intractable. An on-line approximation algorithm is proposed based on Thompson Sampling Algorithm, which acts as heuristic search for best channel in the spectrum.

UR - https://www.scopus.com/pages/publications/84919444269

UR - https://www.scopus.com/pages/publications/84919444269#tab=citedBy

U2 - 10.1109/VTCFall.2014.6966204

DO - 10.1109/VTCFall.2014.6966204

M3 - Conference contribution

AN - SCOPUS:84919444269

T3 - IEEE Vehicular Technology Conference

BT - 2014 IEEE 80th Vehicular Technology Conference, VTC 2014-Fall - Proceedings

T2 - 80th IEEE Vehicular Technology Conference, VTC 2014-Fall

Y2 - 14 September 2014 through 17 September 2014

ER -

Thompson sampling for opportunistic spectrum access with Markovian rewards

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this