[FIXED] HTTP Requests login is not working with the form data I am providing

Issue

NOTE: this is NOT a duplicate. This is the second time I had to post this because it was thought to be a duplicate but the other responses don't solve my particular issue. After correcting my original/previous post to reflect this, it still remained closed. So, here I am asking the same question again since the other one was closed immediately. Please read fully and try to understand my issue before marking it a duplicate or closed.

I am attempting to login to a site whose URL is "https://wedgfl.aquahawk.us/login" but having a hard time ever getting in. I've been using the following code to attempt this but when printing the page content, all I get back is the page without it actually being logged in.

#!/usr/bin/env python3
import requests

with requests.Session() as c:
    url = 'https://wedgfl.aquahawk.us/login'
    USERNAME = '[email protected]'
    PASSWORD = 'MySecretPassword'
    c.get(url)
    #cookie = c.cookies['connect.sid']
    login_data = dict(username=USERNAME, password=PASSWORD, next='/')
    c.post(url, data=login_data, headers={"Referer": "https://wedgfl.aquahawk.us/login"})
    page = c.get('https://wedgfl.aquahawk.us/')
    print(page.content)

Once I am logged in successfully, the landing page should be "https://wedgfl.aquahawk.us" as you can see from my page = c.get('https://wedgfl.aquahawk.us/') part of my code. I thought maybe it was something with my cookie but even disabling/commenting it out has the same effect. Just in case you are wondering, I am indeed using Python 3

I have indeed pulled the form data I need from the page source. Perhaps my issue is that I am not calling the right key info from the form data? Here is what the info is when I view the page source after logging in:

<form id="userAuth" action="/login" method="POST" style="display: none">
  <div id="userAuthUserNameSubtitle" class="x-hidden">User Name or Email:</div>
  <input id="userAuthUserName" type="text" name="username" spellcheck="false" class="x-hidden">
  <div id="userAuthPasswordSubtitle" class="x-hidden">Password:</div>
  <input id="userAuthPassword" type="password" name="password" class="x-hidden">
  <input id="userAuthSubmit" type="submit" value="Sign In" class="x-hidden">
</form>

I've tried it this way as well just to verify if I am actually logging in but still have the same result:

import requests

payload = {
    'username': '[email protected]',
    'password': 'MySecretPassword'
}

import sys

with requests.Session() as c:
    c.post('https://wedgfl.aquahawk.us/login', data=payload)
    r = c.get('https://wedgfl.aquahawk.us/')
    print "Something from the page that only shows after login" in r.content

I even followed the example from this post and still get the same result:

import requests

url = 'https://wedgfl.aquahawk.us/login'
values = {'username': '[email protected]',
        'password': 'MySecretPassword'}

r = requests.post(url, data=values)
print(r.content)

I am either grabbing the wrong value keys from the form data or maybe I need more info in my code from the form data that this page requires in order to login?

Solution

Check this solution, I have added the proper headers

import requests
import json

session = requests.Session()

loginUrl = "https://wedgfl.aquahawk.us/login"
loginHeaders = {
        "Host": "wedgfl.aquahawk.us",
        "Connection": "keep-alive",
        "Accept": "application/json",
        "Origin": "https://wedgfl.aquahawk.us",
        "X-Requested-With": "XMLHttpRequest",
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36",
        "Content-Type": "application/x-www-form-urlencoded; charset=UTF-8",
        "Referer": "https://wedgfl.aquahawk.us/login",
        "Accept-Encoding": "gzip, deflate, br",
        "Accept-Language": "en-US,en;q=0.9",
}

loginData = {
    "username":"[email protected]",
    "password":"MySecretPassword"

}

#Data is posted as Json Data using json.dumps
loginResponse = session.post(loginUrl, data=json.dumps(loginData), headers=loginHeaders)
#If the above does not work try without json.dumps
loginResponse = session.post(loginUrl, data=loginData, headers=loginHeaders)
print(loginResponse)   #should print 200

#check if logged in 

verify = session.get("https://wedgfl.aquahawk.us/")
print(verify.text)

Answered By - Stack