finding Convergence of Automata (an atomaton or more) Algorithm for a and b parameters

This algorithm is proposed using Learning Automata(LA). This method is in fact a new channel allocation process in which LA updates the probability that channels are free in the next time intervals according to channel response. Each Secondary User (SU) has one LA and each LA selects one of the M channels-available in Primary User (PU). Therefore, LA has m operation, namely . LA in SU in the time slot t selects the channel based on probability vector , so that is channel i selection probability in time slot t. After receiving the ACK signal, LA uses a reinforcing model to update probability vector for the next time slot.

This is My Algorithm:

Initialize a and b based on the selected mode: LR-I, LR-P, LReP

LET Pi(1)=1/M for i=1,2,3,… ,M

WHILE not end of iterations t

Select channel i if Pi (t)≥Pj (t) when i≠j. Then sense it (channel i) and assign if it is free,

IF Response = PU_Free % * means multiply %

   Pi(t+1)=Pi(t)+a*[1-Pi(t)]                    
    Pj(t+1)=(1-a)*Pj(t)                   ∀j≠i  

ELSE IF Response = PU_BUSY

   Pi(t+1)=(1-b)*Pi (t)                               
     Pj(t+1)=Pj(t)+b[1/(M-1)-Pj(t)]          ∀j≠i   

END IF

END WHILE

In the LReP model, there is penalty factor with very small value compared to the reward, b very little than a, e.g. b=0.1*a.

In this algorithm:

In the LReP model we have: a=0.09 and b=0.009 .

In the LR-I model, we have: a=0.09 and b=0 .

And in the LR-P, we have a=b=0.09

In the above Algorithm, we use 10 channel. Each channel can be free beta=0 or busy beta=1. Important Questions:

1- If this algorithm converge or dis-converge ?

2- If this convergence related to a and b parameters in each model: LReP , LR-I or in the LR-P?

3- Can you determine boundary of a and b for convergense?

According to [13-14] of LA-based linear methods: Linear Reward-inaction (LR-I), Linear Reward Penalty (LR-P) and Linear Reward-ε-Penalty (LReP) were investigated in general. In accordance with [15] it has provided a scenario with a SU with the initial unknown traffic.

[13] MANDAYAM, A. L., TATHACHAR K. S., NARENDRA Learning Automata: An Introduction. Prentice Hall. 2012.

[14] AART, Fides, VAANDRAGER, Frits Learning I/O Automata. In Paul Gastin and Fran_cois Laroussinie, editors, CONCUR - Concurrency Theory, 2010, vol. 6269 of Lecture Notes in Computer Science, p. 71-85. Springer Berlin / Heidelberg. DOI: 10.1007/978-3-642-15375-4_6.

[15] LAI, L., GEMAL, H., JIANG, H., et al. Optimal medium access protocols for cognitive radio networks. 6th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks and Workshops. 2008. DOI: .10.1109/WIOPT.2008.4586086.