Serious AD help (senior level)

B

bigstusexy

2[H]4U
Joined
Jan 28, 2002
Messages
3,194
Ok so you may remeber me posting about having anew job and how bad AD is and what not.

Now here is a problem that I need serious quick help with. I'm not going to protect pride or crap like that, I don't know what I'm doing! Bad thing here is not only am I paid like crap but the real point is that I know the most out of EVERYONE here about AD

Here is the situation:

I brought up two new server yesterday (2003) and they seemed to enter the domain just fine, one of them was a replacement for a DC that went bad, I was able to remove and scrape it from the Domain just fine, its running with the same IP and name as the old one just different UID, thats no problem. However the second one that I brought up was completely new this one had a few problems I think I have them takent care of. The Huge problem with them is that they don't want to replicate, when I got ot sites and services and click the link and tell it to replicate now it gives me an error : The naming context is in the process of deletion or is not replicated from the specified server.

That makes me think that maybe some DNS/AD info has not propergated to the server that I'm trying to replicate with. Also the fact that at first the Computer/Server accounts were on that DC too. So I got a partial replication from the server that the new ones seemed to have registered with. so now I can see computer accounts in DNS, maybe it was only DNS info thats circulated and not AD The server (call it KMS) that the two I created (WTS1 and 2, 2 its the new one) has not replicated so long that now it will not because of the fact that its been longer than the toombstone life of objects. I tried setting the registry key to force it to do it anyway and its not working so I tried repadmin. I don't know the (common name is it?) format of AD I've dried but it's giving me the error:

Win32 Error 8419(0x20e3): The DSA object could not be found

Now I admit that I don't know what I'm doing but I'm trying. I'm trying to force an entire sync to this DC and have it send out any changes, also I think this DC (KMS) is also the GC holder, so I need to get that changed before I kill it or go nuts.

repadmin /replicate vms01_dns DC=sdsuper DC=local /force /full

Our Domain name is SDsuper.local
Name of the DC I'm trying to replicate with is VMS01

What are you suggestions for this problem, I'll proivde any other needed info.
What am I doing wrong with the repadmin command?


I know the administration side of AD I'd say I'm about junior / entry level of AD I admit this. Believe you me, I'd love to blow away this entire doman and start again but I can't even get them to buy a darned extra IDE controller for one server that had its primary chain killed by our dumb contractors, it would also be too confuing and too much work right now. I don't want to blow away KMS01 because it holds the GC I believe and think that would cause havoc with the domain, not to mention that I'd have to do the same for the WTS01 and 02 because it seems as thought the domain doesn't fully recgonize it, I can't replicate them to KMS because its refusing to replicate. I don't even know how they joined... HELP!
 
I'll take a stab here without looking too much into the errors right now. Was the server you lost (that you replaced) the GC? Try demoting all the servers except for the one server that was there before you brought the other 2 up. Set it as the GC and force it to replecate. Even though it's the only server in the domain see if it errors out. Go though and make sure you remove any assocation to the demoted servers. Then bring the other two back in one at a time. Force replecation and watch it for a while before bringing the next one in.
 
I think I got the repadmin to work after walking away for a few minutes,

its repadmin /replication <fully qualified [client+dns] destination> <full qualified source> Disguinished name (I can't really explan that one I know some but not enought to explain)

It gave me the following error, this mean that it was trying to replicate but here is one of the other main problems is this error:

DsReplicaSync() failed with status 1127 (0x467):
While accessing the hard disk, a disk operation failed even after retries.

I've been getting this error, and I think it has to do wtih NTDS.dit not dll, but still it will not replicate with this error, we dont' have a backup of this file, let me pull the specific error I found:

Event Type: Error
Event Source: NTDS ISAM
Event Category: Database Page Cache
Event ID: 474
Date: 4/29/2005
Time: 10:20:40 AM
User: N/A
Computer: KMSDC02
Description:
NTDS (600) NTDSA: The database page read from the file "C:\WINNT\NTDS\ntds.dit" at offset 59621376 (0x00000000038dc000) for 8192 (0x00002000) bytes failed verification due to a page checksum mismatch. The expected checksum was 3965725494 (0xec602b36) and the actual checksum was 100197752 (0x05f8e578). The read operation will fail with error -1018 (0xfffffc06). If this condition persists then please restore the database from a previous backup. This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem.
For more information, see Help and Support Center at


I've been searching on this error but haven't pulled anything useful. I though forceing a full replication it would send out its changes then completely rewrite its database, but that doesn't seem to be the case :) at least I'm not feeling totally useless now!
 
LittleMe said:
I'll take a stab here without looking too much into the errors right now. Was the server you lost (that you replaced) the GC? Try demoting all the servers except for the one server that was there before you brought the other 2 up. Set it as the GC and force it to replecate. Even though it's the only server in the domain see if it errors out. Go though and make sure you remove any assocation to the demoted servers. Then bring the other two back in one at a time. Force replecation and watch it for a while before bringing the next one in.
Click to expand...
I don't really have the power to do that. I'd have to get convormation etc etc and it would never be resolved. In the future doing the off season is to do just that, Get one DC with the GC and DNS have it nice and sweet and demote everyone else, bring them back up gradually and carefully! See my other response


Thanks for your suggestions.
 
I haven't been back there in a few days I got rid of the other issue and I understand common names... a bit now.

However I still have a problem with the server not doing replications. The problem lies in NTDS.dit there is a checksum error, What do you suggest doing? My thought is to run NTDSutil from DSrecovery mode just on that file, I used the file recovery before but I didn't specify a file.

I don't have a backup of that file in a good state, from the loogs looks like there has been a problem for years and a major problem for about 1 month longer than I have been working here.

Suggestions?
Stu-

P.S. I didn't consider downing and then re promoting the server because its the only one that has the new records for the two servers I just put up, If it goes then I'll have to re register the two other server and I'd like to skip that hassle :)
 
if ntdsutil doesn't work, try esentutl using the transaction logs. Not sure though since you said it's been going on for over a year, worth a try though. You should really check out the drives in the primary server though to make sure they aren't dying.

Edit: KB816120 shows how to use esentutl
 
Thanks littleMe I'll check this out (cause you just lost me on how to do that)

Yeah its a real mess. Here is the thing however that I don't understand. The two new DCs they are up and running and *some of the data has proprigated to the rest of the domain, the domain knows they are there but not all of the info is there. They are refusing to do any new doain duty, like accept new member servers and such (we have a snap server which it will not allow to rejoin the domain and copy the users) that is normal for when it has been so long since it replicated at least of what I know.

What I don't understand is why this server that is causing the problem (with the ntds.dit) hasn't done the same thing? if it had all the mess would be a lot less of a problem because I could just demote and re-promote it.

Thanks again I'm going to go study (for minimum wage grumble grumble)
 
At this point, I'd recommend you call MS and open a ticket. Not cheap but worth it. Unless your worksite has a technet plus subscription, you get 2 free tickets with that. Maybe you could just get them to purchase a year of it ($895) so you get all of technet and 2 support calls.
 
Couple of things: When you say "the" GC server, I'm hoping that you have more than one. Also, if you're in a multi-domain environment, that shouldn't be on the same server as the one that holds the Infrastructure Master role.

Assuming that you have more than once GC now, I'd take the server that's having trouble with the replication and dcpromo it down to a member server and start over with the cleanup of AD. I'd guess that there are still some leftovers from the original server, particularly replication metadata that was never cleaned up. Perhaps the replicating partners still remember the USN where they leftoff with the old server. I'd clean out the directory by doing the dcpromo to demote the server first, then perform the metadata cleanup described here:

http://www.microsoft.com/technet/pr...ons/012793ee-5e8c-4a5c-9f66-4a486a7114fd.mspx

And then I'd give it time to replicate. You should know your environment's "convergence time" which is the maximum time it would take to replicate a change from any DC to all the others.

Good luck. Hope this helps and isn't too redundant from what you've already done.
 
rcolbert said:
Couple of things: When you say "the" GC server, I'm hoping that you have more than one. Also, if you're in a multi-domain environment, that shouldn't be on the same server as the one that holds the Infrastructure Master role.
Click to expand...
Just nitpicking here...with Exchange 2003 in a single domain, it loves to hold the Infrastructure Master FSMO role as well as being a GC.

Solid advice elsewhere. I had to do this last week as well after demoting a machine and blasting it away with ntdsutil and hosing DNS. This was right before I nuked and paved the install and started over. Hate doing that, leaves a nasty taste in my mouth.
 
Everyone's words are useful :) I don't have that much experience at more than just administration of AD, I'd have to have many many computers then be dumb and create to purposely create problems.

Everything in the domain is mostly default,replication time is 1hr I find the errors witht the two new servers when I tell it to replicate via sites and services. Hmm... I wonder if I can force them to replicate now that I know how to use repadmin, if I could get the other server to accep the replication data from them, they should send all their registration information. Its worth a try.

I can't get these cheap bastards to buy jack! Sorry for the language but its true. I get paid minimum wage but besides that. We had out so call "contractors" Fry half of a server's IDE controller, problme is that we had a raid 5 windows voluem on it! 1 parity and 2 stripe. So after I cleared out the "new/temp guy" (thye aren't telling us anything we last heard he was a temp we now think he is the new guy) and the two Contractors,and found out it was broken I turned to the new/temp guy and told him we need to buy a new IDE controller nothing fancy just get the server to work while we move it to the new one... Nope! So I ended up moving the drives around so the stripe set is there but not the parity, windows fails the redundancy after you tell it to initialize the set but the data is accessable. Less than 80 bucks and they would spend it. The system died because the contractors overloaded the power in the room! It was such a gross mistake that I spotted it as soon as I walked into the room

Second stupid thing we had a server die, we have two brand new ones there to take its place. I told we should buy a replacement scsi drive to put in the system, the entire school was complaining about a project that was doo soon and down state so the needed to be back up and running then. When the temp/new guy started back tracking like he wanted to go form the server to the new ones, I told him that since there is no documentation on these and whats on there it will be harder going from a dead one to a new one and it will take longer and there will be errors... what did they decided? To go for dead to new.

So I can't get the to do jack, we don't even get parts to fix the workstations! Sorry for all the ranting.

I was looking (forgot where) but it seemed to have noted that there were several other copies of the GC, I was lead t obelieve that there was only one, I'm not sure though what role this server holds, I'll have to look it up.

To tell you the turth with and minimal experience I have with AD I knew from the moment I touched it that we should/needed to bring down the domain and start again.

*sigh* Thanks for the help keep it comming
Stu-

P.S. I haven't been able to get back to the box physically yet I'll tell you when I do.
 
feigned said:
Just nitpicking here...with Exchange 2003 in a single domain, it loves to hold the Infrastructure Master FSMO role as well as being a GC.
Click to expand...

Not having the Infrastructure Master server being a GC is advice only for a multi-domain environment. The problem it prevents is well documented.

http://www.microsoft.com/technet/pr...elp/9a353810-8e3a-4023-a557-db1a686d8ec8.mspx

Quote = "Unless there is only one domain controller in the domain, the infrastructure master role should not be assigned to the domain controller that is hosting the global catalog. If the infrastructure master and global catalog are on the same domain controller, the infrastructure master will not function. The infrastructure master will never find data that is out of date, so it will never replicate any changes to the other domain controllers in the domain.

In the case where all of the domain controllers in a domain are also hosting the global catalog, all of the domain controllers will have the current data and it does not matter which domain controller holds the infrastructure master role."

But I'm curious in your single domain environment which Exchange server would hold the Infrastrcuture Master Role? In our deployment we wouldn't even consider installing Exchange 2003 on a Domain Controller in the first place. Is this an SBS environment?
 
rcolbert: so in a setup such as his, 2 - 3 DC's, how many GC's would he want?
 
LittleMe:

Its a school district. I'd say which one but I don't want to get into any kind of trouble :) There are three techs. for 11 schools on the network 1 off the network, I'd say its save to say there is an average of at least 150 students at every school, some are less but the larger ones more than make up for it.

This job is starting to all around suck, they don't pay me enough, they don't give me enough ability to do what I need to do... I'm going to stop my ranting right there :)
 
LittleMe said:
rcolbert: so in a setup such as his, 2 - 3 DC's, how many GC's would he want?
Click to expand...

With 3 DC's I'd do this:

DC1 = PDC, RIS, FSMO, Schema Master + GC
DC2 = Infrastructure Master
DC3 = GC

That way you can add a 4th DC at some future point and even if you forget to make it a GC your GC functionality and Infrastructure Master functionality still works. Also, if DC1 crashes you can seize it's roles and rebuild it without having to restore AD. Without the GC on DC3 you'd have to restore rather than seize and fail over + rebuild.

With 2 DC's I'd do this:

DC1 = PDC, RIS, FSMO, Schema Master + GC
DC2 = Infrastructure Master + GC

Then, if I ever added a 3rd DC I would remove the GC from DC2 and create the previous 3 DC scenario.

Bottom line is that I'd design towards failover and fault tolerance rather than restore as a method of AD recovery. I would ideally have at least one DC+GC at another location if at all possible.

Another thought process new to 2003 server is that some people are simply making every DC a GC. The overhead isn't very significant anyway in many cases. The only catch to that is that you have to ensure that you truly make every DC a GC or you'll have the IM problem down the road.

In my experience, even though the MS article I quote doesn't say so explicitly, it seems this issue is only truly relevant in a mult-domain environment, not just a single domain environment with multiple domain controllers. In any case, my 3 DC option above is fine for a single domain model because it provides a minimal level of fault tolerance, and it allows you to expand your forest design later without having to revisit the FSMO or GC roles of the initial 3 servers.
 
Here is our current site layout could might miss a few things

I doubt we will be around to do this (have uncorvered some info that leads us to believe that they want to just have the new guy... no its not incompentence like this, I don't know I'm just angry right now) but if we can Kill our current domain and redo it how would you suggest we do it?


School1 - also this is there the district office is, everything connects to here before going to the interent or another school
4 servers right now one 2 are DCs

S2 and S3
They have two servers for booth school as fiber link into school 2 before it hits the T1 back to school1 school 3 may get its own server but housed at school2

s4 s5 s6
2 Servers for all three fiber links between the buildings

S7 S8 S9
2 server per school all three are different locations each have their own T1

S10
4 servers

s11
1 server

In each of these locations there is at least one computer that does just DC work accept for School 11 and school 9, 11 is really small and 9 is just... not following the plan the rest are following.


I would think the 2 current servers at school one cold have all the important rolls with the server doing the least work to hold the GC, choose two remote server that would be too busy or close to where a lot of traffic would be generated to also have the GC

I'd aslo have the AD level raised to native 2k3
 
bigstusexy said:
I think I got the repadmin to work after walking away for a few minutes,

its repadmin /replication <fully qualified [client+dns] destination> <full qualified source> Disguinished name (I can't really explan that one I know some but not enought to explain)

It gave me the following error, this mean that it was trying to replicate but here is one of the other main problems is this error:

DsReplicaSync() failed with status 1127 (0x467):
While accessing the hard disk, a disk operation failed even after retries.

I've been getting this error, and I think it has to do wtih NTDS.dit not dll, but still it will not replicate with this error, we dont' have a backup of this file, let me pull the specific error I found:

Event Type: Error
Event Source: NTDS ISAM
Event Category: Database Page Cache
Event ID: 474
Date: 4/29/2005
Time: 10:20:40 AM
User: N/A
Computer: KMSDC02
Description:
NTDS (600) NTDSA: The database page read from the file "C:\WINNT\NTDS\ntds.dit" at offset 59621376 (0x00000000038dc000) for 8192 (0x00002000) bytes failed verification due to a page checksum mismatch. The expected checksum was 3965725494 (0xec602b36) and the actual checksum was 100197752 (0x05f8e578). The read operation will fail with error -1018 (0xfffffc06). If this condition persists then please restore the database from a previous backup. This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem.
For more information, see Help and Support Center at


I've been searching on this error but haven't pulled anything useful. I though forceing a full replication it would send out its changes then completely rewrite its database, but that doesn't seem to be the case :) at least I'm not feeling totally useless now!
Click to expand...

Call me crazy, but that sounds like a bad hard drive.. if you can, run chkdsk on it.

==>Lazn
 
rcolbert said:
Not having the Infrastructure Master server being a GC is advice only for a multi-domain environment. The problem it prevents is well documented.

http://www.microsoft.com/technet/pr...elp/9a353810-8e3a-4023-a557-db1a686d8ec8.mspx

Quote = "Unless there is only one domain controller in the domain, the infrastructure master role should not be assigned to the domain controller that is hosting the global catalog. If the infrastructure master and global catalog are on the same domain controller, the infrastructure master will not function. The infrastructure master will never find data that is out of date, so it will never replicate any changes to the other domain controllers in the domain.

In the case where all of the domain controllers in a domain are also hosting the global catalog, all of the domain controllers will have the current data and it does not matter which domain controller holds the infrastructure master role."

But I'm curious in your single domain environment which Exchange server would hold the Infrastrcuture Master Role? In our deployment we wouldn't even consider installing Exchange 2003 on a Domain Controller in the first place. Is this an SBS environment?
Click to expand...
We only have one domain, single forest, single tree. Isn't the Infrastructure role essentially useless? If not, I've been properly owned.

Moving on.

I wish it were SBS or something less stupid. As it is, we have ~100 clients, three DC's currently, two of which are GC's (one being the Exchange box) and one server that does nothing other than back up data which is supposed to be a DC, but I'm going to see if I can let it slide. I actually had it promoted and ready to go, then the other admin demoted it and messed around with some installed software.

The Exchange box I know is a bad idea to even consider when thinking about making it a DC, but while my opinion (which is exactly how you would configure this) is absolutely correct in this aspect, it has no merit in the network that I've helped form and mold. I know it's a bad idea to have it as a DC because it takes 15 minutes just to shut down because the IS service pukes out when A.D. shuts down its services before IS can even touch it. I've told the other admin about this crap and of course it doesn't matter. All servers have to be DCs. I could go on and on about how things should be, but never will be because I don't hold 100% authority. It's something like %40, probably less.

...

Ideally, I'd like to have two 1u boxes that did nothing but hold FSMO roles, AD DNS records, and be DCs. That would leave the database server, Exchange, filserver, and the backup box to their own devices...literally. In a perfect world of course, one that contained an honest-to-god 42u rack. Argh.
 
feigned said:
We only have one domain, single forest, single tree. Isn't the Infrastructure role essentially useless? If not, I've been properly owned.

Moving on.

I wish it were SBS or something less stupid. As it is, we have ~100 clients, three DC's currently, two of which are GC's (one being the Exchange box) and one server that does nothing other than back up data which is supposed to be a DC, but I'm going to see if I can let it slide. I actually had it promoted and ready to go, then the other admin demoted it and messed around with some installed software.

The Exchange box I know is a bad idea to even consider when thinking about making it a DC, but while my opinion (which is exactly how you would configure this) is absolutely correct in this aspect, it has no merit in the network that I've helped form and mold. I know it's a bad idea to have it as a DC because it takes 15 minutes just to shut down because the IS service pukes out when A.D. shuts down its services before IS can even touch it. I've told the other admin about this crap and of course it doesn't matter. All servers have to be DCs. I could go on and on about how things should be, but never will be because I don't hold 100% authority. It's something like %40, probably less.

...

Ideally, I'd like to have two 1u boxes that did nothing but hold FSMO roles, AD DNS records, and be DCs. That would leave the database server, Exchange, filserver, and the backup box to their own devices...literally. In a perfect world of course, one that contained an honest-to-god 42u rack. Argh.
Click to expand...

Exchange slow to shut down on a DC: http://support.microsoft.com/kb/829361
"To make a Windows Server 2003 domain controller computer that has Exchange 2003 installed shut down faster, manually stop the Exchange 2003 services before you shut down Windows." - Specifically, shut down the information store service

Also please note this: http://support.microsoft.com/kb/822179
"You can run Exchange Server 2003 on either a member server or on a domain controller. After you install Exchange Server 2003 on a server, do not change the role of the server. For example, if you install Exchange Server 2003 on a member server, do not use the Dcpromo tool to promote the server to a domain controller. Or, if you install Exchange Server 2003 on a domain controller, do not use the Dcpromo tool to demote the server to a member server. Changing the role of a server after you install Exchange Server 2003 may result in loss of some Exchange functionality and is not supported."

While we are at it, you already got this but I will say it anyways: http://support.microsoft.com/kb/875427
"The domain controller that is running Exchange must also be a global catalog server. "

I ended up with Exchange 2003 on a DC in what was supposed to be a temporary basis.. but then I found out about the above... so it is now stuck a DC untill it is rebuilt/replaced.

==>Lazn
 
feigned said:
We only have one domain, single forest, single tree. Isn't the Infrastructure role essentially useless? If not, I've been properly owned.
Click to expand...

Agreed. I said as much in the bottom paragraph of post #19.



feigned said:
I wish it were SBS or something less stupid. As it is, we have ~100 clients, three DC's currently, two of which are GC's (one being the Exchange box) and one server that does nothing other than back up data which is supposed to be a DC, but I'm going to see if I can let it slide. I actually had it promoted and ready to go, then the other admin demoted it and messed around with some installed software.

The Exchange box I know is a bad idea to even consider when thinking about making it a DC, but while my opinion (which is exactly how you would configure this) is absolutely correct in this aspect, it has no merit in the network that I've helped form and mold. I know it's a bad idea to have it as a DC because it takes 15 minutes just to shut down because the IS service pukes out when A.D. shuts down its services before IS can even touch it. I've told the other admin about this crap and of course it doesn't matter. All servers have to be DCs. I could go on and on about how things should be, but never will be because I don't hold 100% authority. It's something like %40, probably less.

...

Ideally, I'd like to have two 1u boxes that did nothing but hold FSMO roles, AD DNS records, and be DCs. That would leave the database server, Exchange, filserver, and the backup box to their own devices...literally. In a perfect world of course, one that contained an honest-to-god 42u rack. Argh.
Click to expand...


Sounds like you have all the right ideas and it's too bad that your dealing with someone who appears to have some inflexible and pointless rules of thumb (e.g. every server must be a DC.) Time for an I.T. coup IMO. :)
 
Thanks for the links Lazn, I will try and revisit the Exchange on DC issue sometime down the road. Maybe we can be demotion buddies. :p You too rc. Doubt a coup is going to happen but I'd like to change things around a bit.

Did the OP figure out his problem? Sorry for the hijack!
 
No I didn't, I don't mind the hijack really :) I need to get into exchange one day. I've seen a few jobs I could have at least tried for if I knew it.
 
Ok I think I've kinda got something here:

I've been focusing on the wrong file (duc I should have read the Error more closely!) I was foucing on c:\winnt\system32\ntds.dit While the error is for c:\winnt\NTSD\ntds.dit now here is thing, I am not sure where it exist in two places, the other servers only have it in one. Perhaps something from when this server was upgrated (way before my time) the one in NTDS\ is much later than any other ntds.dit file I've seen on the network so for. I'm talking 6 megs as normal vs. this file at 57 megs. I'm not sure how to run esentutl but from what I read on the instructions seems like I can just change to the directory then try a repair command. If I have to specify the other files it talks about I'm sol :)

I still haven't been able to get back to it physically to get it into restore directory services mode. I know I can get it to do that remotely with msconfig (or editing boot.ini in 2k serv) but the box never restarts correctly :) and no one here believes in remote wake up. (woot!)

I'll see if I can get back there this week.
 
You must log in or register to reply here.
Back
Top