flat assembler
Message board for the users of flat assembler.

Index > Heap > Is there an FF add-on for rewriting web pages automatically?

Author
Thread Post new topic Reply to topic
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17462
Location: In your JS exploiting you and your system
revolution
  • I want to rewrite parts of web pages automatically as they are loaded and before they are displayed.
  • Using a regular expression would be best, but search & replace would also be good.
  • Must be able to save the expression(s) to disk.
  • Starts/enabled automatically when the browser starts.
My reason:

Google have recently changed their search result to insert an URL redirector for each link.

eg. Each link in the results now looks like this: https://encrypted.google.com/url?url=<the link url>&rct=j&sa=U&ei=<tracking codes>&ved=<some more tracking codes>&q=<the search query>&usg=<another tracking code>

I am looking for a way too convert the above tracking link into the direct link, i.e just <the link url> only and not all the other surrounding stuff.

Is there such a thing? I searched the add-ons site, found something called firebug that might do it but it seems be be far more bloat than I need it for. And from reading the docs it doesn't state clearly that it will do all that I want.

Any suggestions?
Post 06 Nov 2011, 18:37
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid
Exactly same thing was making me mad. Too bad I use Opera and don't know any sane solution.
Post 06 Nov 2011, 19:29
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4240
Location: 2018
edfed
maybe adblock system can be a good basis. as it search for patterns to ignore, the same algorithm can be used to replace patterns by others.
Post 06 Nov 2011, 19:39
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17462
Location: In your JS exploiting you and your system
revolution
I found redirect cleaner: https://addons.mozilla.org/en-US/firefox/addon/redirect-cleaner/reviews/ by using google to search the FF add-ons page. Such irony! Anyhow, I haven't tried it yet. Anyone here has tried it? Or know of a better option?

[edit] It can't be installed in 3.6.23, incompatible version and no update is available.
Post 06 Nov 2011, 19:59
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7751
Location: Kraków, Poland
Tomasz Grysztar
vid wrote:
Exactly same thing was making me mad. Too bad I use Opera and don't know any sane solution.

In Opera you can have your own JavaScript executed in every loaded page, so it should be easy to do such thing with it.
Post 06 Nov 2011, 20:40
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid
I am aware of this feature, but I can't write such a thing myself and can't find anything existing.
Post 06 Nov 2011, 20:45
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7751
Location: Kraków, Poland
Tomasz Grysztar
vid wrote:
I am aware of this feature, but I can't write such a thing myself and can't find anything existing.

The "Clear URI" script from this page: http://www.howtocreate.co.uk/operaStuff/userJavaScript.html
seems to do the thing.
Post 06 Nov 2011, 21:38
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid
Owww crap, I just realized I turned off JavaScript for google.com, in order to get rid of their ultra-annoying features like catching arrow keys, or catching letter keys into search query even when it doesn't have keyboard focus. Thanks a lot for your effort, but until there is some userJS which blocks that, I'll rather have links mangled Sad

[edit]hmm, "Stop form focus" seems interesting
[edit2]Nay, "Stop form focus" doesn't seem to stop Google form focus Sad


Last edited by vid on 06 Nov 2011, 22:12; edited 1 time in total
Post 06 Nov 2011, 21:57
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17462
Location: In your JS exploiting you and your system
revolution
It used to be that google modified the direct link upon an onmousedown event with JS on the fly and many old redirect-remover scripts (like in greasemonkey) would just remove the onmousedown field. But in the last two days that has now changed, the google server will specify every link with the redirected name. This is why vid and myself only now see the problem, previously with JS disabled we would never see it.
Post 06 Nov 2011, 22:05
View user's profile Send private message Visit poster's website Reply with quote
Matrix



Joined: 04 Sep 2004
Posts: 1171
Location: Overflow
Matrix
hehe, i have done it few days ago, it was annoying.

1: i recommend you linux Wink

2: install squid

3: configure it to not post squid headers, be transparent

4: make an url_rewrite_program (or use) to rewrite the url address

for example i did:

rewrite0.sh :
Code:
#!/bin/bash

unescape()
{
while IFS= read -d '' -r -n1 c; do [[ $c = % ]] && { read -n2 hex; printf \\x"$hex"; } || printf '%s' "$c"; done
}

while read teststr; do
if [ `printf '%s' "$teststr" | grep -Eic 'google.[^/\.]+/url?'` -gt 0 ];then
#format opera:
#[HTTP_USER_AGENT] = "Opera/9.80 (X11; Linux x86_64; U; en) Presto/2.7.62 Version/11.01"
#base url:
#http://www.google.hu/search?sclient=psy-ab&hl=hu&client=opera&rls=en&channel=suggest&biw=1024&bih=661&noj=1&source=hp&q=123&oq=123&aq=f&aqi=&aql=&gs_sm=e&gs_upl=2793738l2794160l0l2794318l3l2l0l0l0l0l0l0ll0l0
#first hit:
#http://www.google.hu/url?sa=t&rct=j&q=123&source=web&cd=1&ved=0CCUQFjAA&url=http%3A%2F%2Fwww.123greetings.com%2F&ei=rMarTtOQHpPE4gSAi9njDg&usg=AFQjCNFL87MiXYWoD2vDUTH3ti_WejzGkA
#destination:
#http://www.123greetings.com/
#teststr="http://www.google.hu/url?sa=t&rct=j&q=123&source=web&cd=1&ved=0CCUQFjAA&url=http%3A%2F%2Fwww.123greetings.com%2F&ei=rMarTtOQHpPE4gSAi9njDg&usg=AFQjCNFL87MiXYWoD2vDUTH3ti_WejzGkA"

#opera useragent not mentioned setting opera:config: mozilla
#[HTTP_USER_AGENT] = "Mozilla/4.0 (compatible; MSIE 8.0; X11; Linux x86_64; en) Opera 11.01"
#http://www.google.hu/url?q=http://www.asdf.com/&sa=U&ei=htKrTrrmArOP4gTJrpDaDg&ved=0CBUQFjAA&usg=AFQjCNFAHcf8KWid6S6QNJnoVJLmgaq_JA
# =>http://www.asdf.com/
#teststr="http://www.google.hu/url?q=http://www.asdf.com/&sa=U&ei=htKrTrrmArOP4gTJrpDaDg&ved=0CBUQFjAA&usg=AFQjCNFAHcf8KWid6S6QNJnoVJLmgaq_JA"
#teststr="http://www.google.hu/url?q=http://www.fdsfranchise.com/&sa=U&ei=VNarTtP_JeXT4QTv-ezhDg&ved=0CCEQFjAF&usg=AFQjCNHvzRlU27Dnf8her-W-UBORYw8ZtQ"
printf '%s\n' "$teststr" | sed -E 's;.*(\&|\?)url=([^ &]+).*|.*(\&|\?)q=([^ &]+).*;\2\4;' | unescape | sed 's;.*;303:&;'
printf '%s => ' "$teststr" >> /var/log/squid/rewrite.log;printf '%s => %s\n' "$teststr" | sed -E 's;.*(\&|\?)url=([^ &]+).*|.*(\&|\?)q=([^ &]+).*;\2\4;' | unescape | sed 's;.*;303:&;' >> /var/log/squid/rewrite.log
else
printf '%s\n' "$teststr" | tee -a /var/log/squid/rewrite.log
fi
done
    


its in bashscript, in opera still fails to reload redirect for some reason even if i replace the address correctly if i don't set opera:config useragent spoof to 4 that makes mozilla http-user-agent.

or maybe you can try using a greasemonkey (plugin) script for example, though you know urls can be hidden several ways, it may be easier to replace the actual query in this case...
Post 07 Nov 2011, 00:30
View user's profile Send private message Visit poster's website Reply with quote
ctl3d32



Joined: 30 Dec 2009
Posts: 204
Location: Brazil
ctl3d32
You can use GreaseMonkey. It's a firefox addon.

Description: Allows you to customize the way a web page displays or behaves, by using small bits of JavaScript.

Link: https://addons.mozilla.org/pt-br/firefox/addon/greasemonkey/
Post 07 Nov 2011, 02:48
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17462
Location: In your JS exploiting you and your system
revolution
ctl3d32: JS is not a solution for me.
Post 07 Nov 2011, 03:20
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17462
Location: In your JS exploiting you and your system
revolution
I have a partial solution, it is a bit hacky.

Install https://addons.mozilla.org/en-US/firefox/addon/customize-your-web/

Stage 1:
  • Make a new definition for: https://encrypted.google.com/search?*
  • Set target to : //HTML/body
  • Set attribute: display to value: block
[This allows the first page with the normal links to be displayed (block mode instead of none). Without this the results are hidden by a noscript tag that sets the display value to none]

But we are not done yet. You will notice the page "refreshes" shortly after being displayed and you get the mangled URLs.

Stage 2:

In NoScript (another add-on which you will all already have) set the option under the advanced tab to "Forbid META redirections inside <NOSCRIPT> elements". You might also want to turn off the notification message so you don't get the annoying message bar.

[This will stop the page from redirecting you to the server side URL rewriter that adds the URL redirector links]

Phew. Normal links, no redirections, no refreshes, no annoying notification bars. Cool
Post 07 Nov 2011, 05:59
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid
Quote:
Phew. Normal links, no redirections, no refreshes, no annoying notification bars.

And no link to "Cached" version of page I presume?
Post 07 Nov 2011, 10:18
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17462
Location: In your JS exploiting you and your system
revolution
vid wrote:
And no link to "Cached" version of page I presume?
Nope. What is that useful for anyway? It also sends you to google's server.
Post 07 Nov 2011, 10:26
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid
Of course, but I often need to access the cached version (to circumvent required registration, find cached information on ever-changing page, see contents of already dead site, etc.).
Post 07 Nov 2011, 10:49
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on YouTube, Twitter.

Website powered by rwasa.