something is wrong with Months and Days standardized columns
I’m working through the steps in the videos on my own. I got different values for the “month “ and “Day of the week“
columns when I applied the CustomScaler class, but the rest of the rows have same values as the video.
So, I thought maybe my “months “and “days” data are different. that’s why they got standardized in
a different way.
I tried to tackle the problem from my preprocessed_data notebook, downloaded the “Final Remarks
on the Data Preprocessing Part of the Exercise – lectures” notebook provided in
“Preprocessing the ‘absenteeism_data’” part, and compared it to my notebook.
Everything looked good till I reached the month extraction process.
The value of “Month” column I got from the extraction are different from the ones in “Final Remarks
on the Data Preprocessing Part of the Exercise – lectures” notebook
On the right is my notebook, and on the left the “Final Remarks
on the Data Preprocessing Part of the Exercise – lectures”
Obviously, there is difference in the data
So, I further investigated and found out that this conflict
was because of an error you didn’t remove, this error was introduced previously
When I removed this cell from your notebook, the date and month data became same as mine.
To double-check that the results in the video were produced from this error, I left the error as it was and went through the whole process again till I reached the CustomScaler and got the same results as the one in the video. This means that the results on the video are faulty.
Hi Hager!
Thanks for reaching out and for taking so much time and patience to check out for such an issue, as well as write us a detailed question and feedback.
Can you please assist us a little more, to make sure we are all on the same page.
- First, I see you are referring to the following lecture from the ML section of the course:
https://learn.365datascience.com/courses/sql-tableau-python/creating-a-custom-scaler-to-standardize-only-numerical-features/
Can you please confirm this? - Second, the screenshots you've provided seem to be from the cleaning and preprocessing part of the course. If that is the case, can you please look at the following few code cells where we actually specify the mode in which we'd like the values from the 'Date' column to be imported? You can look at the month values and see that they are not always ordered in ascending order.
3. If necessary, please do not hesitate to support the discussion with links to the lectures in question.
Looking forward to your answer.
Thank you very much.
Kind regards,
Martin
Hi Martin,
first of all, I'm really grateful for this course, it introduced me to several great techniques and new approaches I never knew about before.
your way of explaining was very professional and simple. despite some difficulties I've faced following along, I really enjoyed this course thank you very much for putting such effort into this course.
1- yes, I was referring to this lecture just to acknowledge you how I found that my results are different from the ones on the video
https://learn.365datascience.com/courses/sql-tableau-python/creating-a-custom-scaler-to-standardize-only-numerical-features/
2- exactly that's the case I’m referring to,
months in your screenshot are not ordered, not in the same order as the original data we're converting from.
here is the date order from the original data
and since we're working with regression. I thought we'd need all columns to be related to each other so that each row represents a single record.
thank you for taking the time and viewing my comment it's an honor talking to you in person.
Hi Hager!
Thank you very much for the kind words. They are touching and only motivating us to keep doing what we do, which is provide courses containing techniques that are valid and useful for the Python user.
What you are referring to is indeed an issue that we were supposed to have dealt with during the preprocessing part of the course, not the ML part. So, we will double-check if that issue is tackled through the creation of the absenteeism module at the current version of the course. Please allow us to take some time to do this and we'll get back to you.
Thank you for the understanding.
Kind regards,
Martin
Hi everyone!
I hope you don't mind if I join the conversation.
We are currently working to resolve the issue. Can you please confirm/send us a link precisely towards the video where you observe the discrepancy you’ve mentioned, so that we can be sure we are all on the same page or, if necessary, work towards correcting the video material provided. Thank you!
Looking forward to your answer.
Kind regards,
Tsvetelin