Forged Alliance Forever

Re: No shit, people

2013-01-07T16:24:13+02:00

Ze_PilOt wrote:
I don't think you realize the amount of log it produce. (About 1 megabyte/second, probably 5 in peak time, not talking of the major server slowdown).

you can try to attach it to the process after it hangs. that should at least show you what the last syscall was for each thread.

strace -f -p

if that doesnt help, you can try to attach it while the server is still healthy with:

strace -f -p 2>&1 | split -l 100000 - tracelog

and remove old tracelog.* files to limit the log size. requires some babysitting.
or:
strace -f -p 2>&1 | perl -e '$SIG{INT} = sub { print @log }; while () { push @a, $_; if (@a > 100000) { unshift @a } }'

or:
run the strace inside screen(1) and let it write to stderr then use can use the line buffer to inspect only the last few hundred lines ("ctrl+a [" and use cursor keys)
about the extra overhead, nothing can be done.

Statistics: Posted by rootbeer23 — 07 Jan 2013, 16:24

Re: No shit, people

2013-01-07T14:46:11+02:00

I don't think you realize the amount of log it produce. (About 1 megabyte/second, probably 5 in peak time, not talking of the major server slowdown).

Statistics: Posted by Ze_PilOt — 07 Jan 2013, 14:46

Re: No shit, people

2013-01-07T13:31:18+02:00

Ze_PilOt wrote:
The server is trying to complete a function but is unable to, so it's freeze completely.
It's not eating the whole memory or the cpu, that make the bug even more strange.

so then run strace on it. then you will see which syscall is being executed and not completed.

Statistics: Posted by rootbeer23 — 07 Jan 2013, 13:31

Re: No shit, people

2013-01-07T13:21:58+02:00

rootbeer23 wrote:
Ze_PilOt wrote:The bad news are that what make the server hung is a bug

what exactly does "hung" mean? is the server not doing network I/O at that point? is it eating up the CPU 100%?
have you tried running strace?

Ze_PilOt wrote:
(2x 1 To in raid and not a single SSD : We don't have HD time access problems, while backuping all the maps and replays is very important).

a raid is not a backup. it does not protect against filesystem corruption nor against software accidently erasing data or
an administrator who runs rm in the wrong directory.

hung
c. (of a situation) unable to be resolved

The server is trying to complete a function but is unable to, so it's freeze completely.
It's not eating the whole memory or the cpu, that make the bug even more strange.

We have a separate backup.
I mean backup in the sense that if an HDD is fried, the server will keep running without any trouble, and the restoration will be faster.

Statistics: Posted by Ze_PilOt — 07 Jan 2013, 13:21

Re: No shit, people

2013-01-07T13:17:59+02:00

Ze_PilOt wrote:
The bad news are that what make the server hung is a bug

what exactly does "hung" mean? is the server not doing network I/O at that point? is it eating up the CPU 100%?
have you tried running strace?

Ze_PilOt wrote:
(2x 1 To in raid and not a single SSD : We don't have HD time access problems, while backuping all the maps and replays is very important).

a raid is not a backup. it does not protect against filesystem corruption nor against software accidently erasing data or
an administrator who runs rm in the wrong directory.

Statistics: Posted by rootbeer23 — 07 Jan 2013, 13:17

Re: No shit, people

2013-01-07T00:13:54+02:00

There will be a server update very soon to be able to keep up with Galactic War.
The current server is enough for our actual need, but it's on the verge on collapsing if I had something else on it

Not ETA on that, I will have to sort out how we will make the transition. But next expiration date on the current one is 27/02. So probably before.

The good news are that the server will be fast at peak time, and we will be able run run additional services.

The bad news are that what make the server hung is a bug, and it's not related to the hardware. It need to be fixed, changing the server won't help. The other bad news is that it cost more than the actual server. While it will be all right for a while, the cost can be a problem after a while. But don't worry yet about it, it's not a real problem before some months, probably a year

The new server will be this one :
https://www.firstheberg.com/dedies/deta ... 0-hdd-2013
(2x 1 To in raid and not a single SSD : We don't have HD time access problems, while backuping all the maps and replays is very important).

Statistics: Posted by Ze_PilOt — 07 Jan 2013, 00:13

Re: No shit, people

2013-01-06T23:34:44+02:00

I appreciate the work and donated a small amount to the server fund in response. I also would be willing to test any lobby/game updates prior to them going live so we do not have this problem again though.

Statistics: Posted by Ionic — 06 Jan 2013, 23:34

Re: No shit, people

2013-01-06T22:59:08+02:00

Well when you see guys like combo complaining because Zep is trying to FIX a bug, and because it's the week end ...enough said!

Dunno about you but fixing BS bugs during the weekend wouldn't be my definition of fun !

So please acknowledge the dedication. Thank you!

Statistics: Posted by -_V_- — 06 Jan 2013, 22:59

No shit, people

2013-01-06T21:50:50+02:00

Hey guys, this is to all of you
today i followed something that i dont want to see ever again.

Zep was trying to get rid of lobby-crashing. An effort worth a thanks at any point.
i dont 100% understand what was the idea, it was running down to make everyone use router port 6112 ONLY.

i was trying to help some people, was reading their logs. Now i can say that most of you ARE NOT using 6112 only.

result was a shitlot of complainers all around the chat.
because some of you were to, i dont know, retared? to read the wiki, where it was explained under connection problems and issues,
zep was pissed of because noone was listening to what he said.

in order to get silence zep reverted the change

so we all will keep crashing lobbys
thx to anyone of you who was too stupid or lazy to READ.
i´m pissed off, seriously!
this needs to be changed in our behaviour, or faf will be dead pretty soon i´m afraid

greetings, brainfart

Statistics: Posted by Master_Vallex — 06 Jan 2013, 21:50