Algorithms that affect student scoring

Does anyone else feel we should know how these algorithms are calculated.

I’ve just been told this by email.

'the scores are worked out by an algorithm, we’re not able to give specifics on exactly how your scores will change ’

I thought I would post it here so Rachel could check its correct. The last time I was told something by support it was completely incorrect. So perhaps you could comment Rachel.

I think we have covered the scope of the repeat client score thread

But please comment or like so we could perhaps get some momentum in trying to get this addressed.

1 Like

The algorithm needs changing in my book, it was fine when it was face-to-face but it causes a lot of anxiety now. Tutorful need to address this directly for sure.

Agreed.

First off it absolutely should not include the free video chats, but only fully booked and paid for lessons.

Second it should only apply on a longer term format. Have students failed to book follow up lessons, but accounted for maybe 2 weeks after the initial booking. Also, if that student then books lessons later on it should instantly revert to a score as if they had booked immediately after the 1st lesson. I’ve had students in the past who had an initial lesson in June and then asked to start in September, but waited until September to book so that they would know what their timetable was.

Third, there should be some sort of feedback from both student and tutor about why lessons weren’t continued. For instance, a student who had all their problems sorted and was perfectly happy shouldn’t count against the repeat booking score. I had one such student a few years ago, he took 4 lessons and wrote me a glowing 5 star review.

Fourth (and I’m not sure this doesn’t already apply) it should be moderated by the total number of students you’ve taught over the long term. Someone who has has had one unhappy student out of 40 in 3 years should not be as badly affected as someone with one unhappy student out of 5 in 1 year.

3 Likes

I think part of the issue is that once we know how it’s calculated then we will adjust our behaviour to maximise our scores. This defeats the point of having them at all because, as Goodhart’s Law says, any measure that becomes a target is no longer a good measure.

More specifically, if we don’t know how it’s calculated then it affects us all equally so no one is at a particular advantage or disadvantage. I’ve been indifferent to the method behind the scenes for this reason, since if everyone is affected and it’s only used to compare everyone then it shouldn’t really make any difference.

However, the fact that free calls effectively can only count against your score seems to be a serious oversight. It may just be Goodhart’s Law in action, but I am certainly concerned by this and considering stopping them. There’s a definite bias in favour of those not offering them, which seems to be the opposite of what everyone wants, Tutorful included.

My overall point is that these are things I want to see addressed, but when they are I think Tutorful shouldn’t give too much info on exactly how it’s calculated. I think it would be best if there was a single combined score shown with some advice on how to improve it, or tutors who are severely affected by a specific area getting contacted automatically to help address it. But whatever happens, then assuming the algorithm is fair then if we know too much then it becomes almost an obligation to ‘game’ the system to remain competitive.

Such a good point. Incredible that we’re penalised for *checks notes* teaching people well enough that they no longer need lessons (or as many as they thought they would).

@Dylan but if we dont know how its calculated how do we do we know its being done correctly .

The losing points when you take on a new student is madness. Someone has come up with this idea and its incorrect.

It’s like i said in the other thread it’s like taking a test and getting zero until its marked.

You go from 100 per cent to zero back to 100. When does that ever happen.

@Rachel_Tutorful said that the score out of 10 is the number of students that will return for more lessons.
That’s definitely not the result the statistic is giving .

But it might be what it’s supposed to give. In which case its incorrect

Thanks to Dylan and others for sharing information about the algorithm. This explains where I have being going wrong recently.

In my experience the new classroom only works in around 80% of cases. When it does not work in a first lesson I have the pain of refunding for a time consuming prepared lesson and Tutorful suffer the pain of reputational damage.

To avoid these problems I had been insisting on a free video chat before accepting a first lesson with a new pupil. That is I have put in unpaid overtime to help Tutorful’s public image. In return Tutorful’s omnipotent algorithm has docked me Brownie points pushed me down the league tables and reduced my income.

It won’t happen again.

@davemillerwitney My point is more that ideally we should be able to trust that the algorithm is sensible and working as intended, and once (if?) that is true we should be able to focus on tutoring instead of getting a top score.

@Philip you’re welcome, just so you know Tutorful should cover the lesson if there is a problem with the classroom. They will want to know that it wasn’t a technical issue at your or the student’s end, but once they know the issue lies with them I have always found they’re good at covering it with little fuss. You shouldn’t have to go without being paid for the session if there isn’t a problem with your own connection. If there’s a problem with the student’s connection then whether or not to refund it is up to you basically, but strictly speaking they are responsible.

Sure I understand that but it’s currently misreporting the likelihood of someone booking and having regular lessons.

This in turn is affecting the likelihood of new clients finding you on the site.

This obviously has no affect on tutorful. They are just as likely to get the commission on a lesson from any tutor chosen

To be honest it’s not affecting me that much because I am now close to capacity anyway.

But as a mathematician it frustrates me that it is so misleading

Im not an economist so hadnt heard of this before. I openly admit to looking it up. So I’ve learnt something today.

But this according to my reading is more of the viewpoint of Marilyn’ Straterns

I’m.not sure it even applies here, I understand the concept.

But how would we be able to affect the statistic. Offer the 1st 3 4 5 6 (however many lessons tutorful say need to be reached to counter the repeat score dip) at the minimum fee.

This is possible. But I cant see many tutors giving up a large portion of their income just to get more students and again having to give up a high proportion of that income.

The cycle would be endless and result in a massive downturn in turnover until all these lessons reached full price.

It’s an interesting point and thanks to Dylan for teaching me something new today.