[ML.1] AML - Reinforcement Learning assignment typos

I would like to start a new blog series - Machine Learning [ML], as I am slowly re-qualifying from pure science to data science.

Recently I finished the fourth course on Coursera in a row from Advanced Machine Learning specialization - Practical Reinforcement Learning.

I will copy my review of the course (mark 5/5):
"This is my fourth AML course, and for now I would say it is the best one. It connects lectures and practice in the best way. On the other hand, there are mistakes all around, as it is beta-version. In my opinion, it is not fair to put the beta-version course into paid specialization."

Yes, you read it right - it is the beta-version. Hence, a few practical tasks (or assignments), had some mistakes where they should not be, i.e. in the part of the code that should not be changed by the student. I will list all of them here and try to make somebodies life easier :)

Week 1 / Assignment 1 - "OpenAI Gym":

There is a wrong hint:
Hint: your action at each step should depend either on t or on s.
One should only use t!!!

Week 6 / Assignment 8 - "Bandirs & exploration":

Change 1 - class BernoulliBandit:

class BernoulliBandit:

def pull(self, action):

line

if np.random.random() > self._probs[action]:

changed to:

if np.any(np.random.random() > self._probs[action]):

i.e. the condition is put under np.any().

Change 2 - def plot_regret:

def plot_regret(scores):

line

plt.legend([agent.name for agent in scores])

changed to

plt.legend([agent for agent in scores])

i.e. agent.name is changed to solely agent.

Change 3 - submission:

Instead of submit_bandits, make new function submit_bandits2 in submit.py:

def submit_bandits2(agents, scores, email, token):
    epsilon_greedy_agent = None
    ucb_agent = None
    thompson_sampling_agent = None
    for agent in agents:
        if "EpsilonGreedyAgent" in agent.name:
            epsilon_greedy_agent = agent.name
        if "UCBAgent" in agent.name:
            ucb_agent = agent.name
        if "ThompsonSamplingAgent" in agent.name:
            thompson_sampling_agent = agent.name
    assert epsilon_greedy_agent is not None
    assert ucb_agent is not None
    assert thompson_sampling_agent is not None
    grader = grading.Grader("VL9tBt7zEeewFg5wtLgZkA")
    grader.set_answer("YQLYE", (int(scores[epsilon_greedy_agent][int(1e4) - 1]) - int(scores[epsilon_greedy_agent[int(5e3) - 1])))
    grader.set_answer("FCHOZ", (int(scores[epsilon_greedy_agent][int(1e4) - 1]) - int(scores[ucb_agent][int(1e4) - 1])))
    grader.set_answer("0JWHl", (int(scores[epsilon_greedy_agent][int(5e3) - 1]) - int(scores[ucb_agent][int(5e3) - 1])))
    grader.set_answer("4rH5M", (int(scores[epsilon_greedy_agent][int(1e4) - 1]) - int(scores[thompson_sampling_agent][int(1e4) - 1])))
    grader.set_answer("TvOqm", (int(scores[epsilon_greedy_agent][int(5e3) - 1]) - int(scores[thompson_sampling_agent][int(5e3) - 1])))
    grader.submit(email, token)